Skip to content

Commit

Permalink
add svg and reference in blog
Browse files Browse the repository at this point in the history
Signed-off-by: Anish Ramasekar <anish.ramasekar@gmail.com>
  • Loading branch information
aramase committed Apr 3, 2023
1 parent 164f7d0 commit 9dc2c6d
Show file tree
Hide file tree
Showing 5 changed files with 89 additions and 26 deletions.
111 changes: 85 additions & 26 deletions content/en/blog/_posts/2023-04-11-kmsv2-beta.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ date: 2023-04-11
slug: kms-v2-moves-to-beta
---

**Authors:** TBD
**Authors:** Anish Ramasekar, Mo Khan, and Rita Zhang (Microsoft)

With Kubernetes 1.27, we are moving Key Management Service (KMS) v2 API to beta.

Expand All @@ -14,44 +14,61 @@ One of the first things to consider when securing a Kubernetes cluster is encryp

KMS v1 has been a feature of Kubernetes since version 1.10, and is currently in beta as of version v1.12. KMS v2 was introduced as alpha in v1.25.

{{< note >}} The design in [the previous blog post](https://kubernetes.io/blog/2022/09/09/kms-v2-improvements/) is not compatible with the design in this blog post. The KMS v2 API and implementation changed in incompatible ways in-between the alpha release in v1.25 and the beta release in v1.27. Attempting to upgrade from old versions with the alpha feature enabled will result in data loss. {{< /note >}}
{{% alert title="Note" color="primary" %}}
The KMS v2 API and implementation changed in incompatible ways in-between the alpha release in v1.25 and the beta release in v1.27. The design of KMS v2 has changed since [the previous blog post](https://kubernetes.io/blog/2022/09/09/kms-v2-improvements/) was written and it is not compatible with the design in this blog post. Attempting to upgrade from old versions with the alpha feature enabled will result in data loss.
{{% /alert %}}

## What’s new in `v2beta1`?
The KMS encryption provider uses an envelope encryption scheme to encrypt data in etcd. The data is encrypted using a data encryption key (DEK). The DEKs are encrypted with a key encryption key (KEK) that is stored and managed in a remote KMS. With KMS v1, a new DEK is generated for each encryption. With KMS v2, a new DEK is generated on server startup and when the KMS plugin informs the API server that a KEK rotation has occurred.
The KMS encryption provider uses an envelope encryption scheme to encrypt data in etcd. The data is encrypted using a data encryption key (DEK). The DEKs are encrypted with a key encryption key (KEK) that is stored and managed in a remote KMS. With KMS v1, a new DEK is generated for each encryption. With KMS v2, a new DEK is only generated on server startup and when the KMS plugin informs the API server that a KEK rotation has occurred.

{{< caution >}} If you are running virtual machine (VM) based nodes that leverage VM state store with this feature, you must not use KMS v2.
{{% alert title="Caution" color="warning" %}}
If you are running virtual machine (VM) based nodes that leverage VM state store with this feature, you must not use KMS v2.

With KMS v2, the API server uses AES-GCM with a 12 byte nonce (8 byte atomic counter and 4 bytes random data) for encryption. The following issues could occur if the VM is saved and restored:

The counter value may be lost or corrupted if the VM is saved in an inconsistent state or restored improperly. This can lead to a situation where the same counter value is used twice, resulting in the same nonce being used for two different messages.
If the VM is restored to a previous state, the counter value may be set back to its previous value, resulting in the same nonce being used again.
Although both of these cases are partially mitigated by the 4 byte random nonce, this can compromise the security of the encryption. {{< /caution >}}
Although both of these cases are partially mitigated by the 4 byte random nonce, this can compromise the security of the encryption.
{{% /alert %}}

### Sequence Diagram

#### Encrypt Request

<!-- source
```mermaid
%%{init:{"theme":"neutral", "sequence": {"mirrorActors":true},
"themeVariables": {
"actorBkg":"royalblue",
"actorTextColor":"white"
}}}%%
sequenceDiagram
participant user
participant kube_api_server
participant kms_plugin
participant external_kms
alt Generate DEK at startup/KEK rotation
kube_api_server->>kube_api_server: generate DEK
kube_api_server->>kms_plugin: encrypt request
kms_plugin->>external_kms: encrypt DEK with remote KEK
external_kms->>kms_plugin: encrypted DEK
kms_plugin->>kube_api_server: return encrypt response <br/> {"ciphertext": "<encrypted DEK>", key_id: "<remote KEK ID>", "annotations": {}}
alt Generate DEK at startup
Note over kube_api_server,external_kms: Refer to Generate Data Encryption Key (DEK) diagram for details
end
user->>kube_api_server: create/update resource that's to be encrypted
kube_api_server->>kube_api_server: encrypt resource with DEK
kube_api_server->>etcd: store encrypted object
```
-->

{{< figure src="/images/blog/2023-04-11-kubernetes-1.27-kmsv2-beta/kubernetes-1.27-encryption.svg" alt="Sequence diagram for KMSv2 beta Encrypt" class="diagram-large" >}}

#### Decrypt Request

<!-- source
```mermaid
%%{init:{"theme":"neutral", "sequence": {"mirrorActors":true},
"themeVariables": {
"actorBkg":"royalblue",
"actorTextColor":"white"
}}}%%
sequenceDiagram
participant user
participant kube_api_server
Expand All @@ -71,37 +88,79 @@ sequenceDiagram
kube_api_server->>kube_api_server: decrypt resource with DEK
kube_api_server->>user: return decrypted resource
```
-->

{{< figure src="/images/blog/2023-04-11-kubernetes-1.27-kmsv2-beta/kubernetes-1.27-decryption.svg" alt="Sequence diagram for KMSv2 beta Decrypt" class="diagram-large" >}}

#### Status Request

<!-- source
```mermaid
%%{init:{"theme":"neutral", "sequence": {"mirrorActors":true},
"themeVariables": {
"actorBkg":"royalblue",
"actorTextColor":"white"
}}}%%
sequenceDiagram
participant kube_api_server
participant kms_plugin
participant external_kms
loop every minute
alt Generate DEK at startup
Note over kube_api_server,external_kms: Refer to Generate Data Encryption Key (DEK) diagram for details
end
loop every minute (or every 10s if error or unhealthy)
kube_api_server->>kms_plugin: status request
kms_plugin->>external_kms: validate remote KEK
external_kms->>kms_plugin: KEK status
kms_plugin->>kube_api_server: return status response <br/> {"healthz": "ok", key_id: "<remote KEK ID>", "version": "v2beta1"}
alt KMS plugin not healthy ("healthz" != "ok")
loop every 10 seconds until healthy (blocking)
kube_api_server->>kms_plugin: status request
end
end
alt KEK rotation detected
alt Rotate DEK
kube_api_server->>kube_api_server: generate new DEK
kube_api_server->>kms_plugin: encrypt request
kms_plugin->>external_kms: encrypt DEK with remote KEK
external_kms->>kms_plugin: encrypted DEK
kms_plugin->>kube_api_server: return encrypt response <br/> {"ciphertext": "<encrypted DEK>", key_id: "<remote KEK ID>", "annotations": {}}
end
alt KEK rotation detected (key_id changed), rotate DEK
Note over kube_api_server,external_kms: Refer to Generate Data Encryption Key (DEK) diagram for details
end
end
```
-->

{{< figure src="/images/blog/2023-04-11-kubernetes-1.27-kmsv2-beta/kubernetes-1.27-status.svg" alt="Sequence diagram for KMSv2 beta Status" class="diagram-large" >}}

#### Generate Data Encryption Key (DEK)

<!-- source
```mermaid
%%{init:{"theme":"neutral", "sequence": {"mirrorActors":true},
"themeVariables": {
"actorBkg":"royalblue",
"actorTextColor":"white"
}}}%%
sequenceDiagram
participant kube_api_server
participant kms_plugin
participant external_kms
kube_api_server->>kube_api_server: generate DEK
kube_api_server->>kms_plugin: encrypt request
kms_plugin->>external_kms: encrypt DEK with remote KEK
external_kms->>kms_plugin: encrypted DEK
kms_plugin->>kube_api_server: return encrypt response <br/> {"ciphertext": "<encrypted DEK>", key_id: "<remote KEK ID>", "annotations": {}}
```
-->

{{< figure src="/images/blog/2023-04-11-kubernetes-1.27-kmsv2-beta/kubernetes-1.27-generate-dek.svg" alt="Sequence diagram for KMSv2 beta Generate DEK" class="diagram-large" >}}

### Performance Improvements


## What's next?
For Kubernetes v1.28, we expect the feature to stay in beta. In the coming releases we want to investigate:
- Cryptographic changes to remove the limitation on VM state store.
- Kubernetes rest API changes to enable a more robust story around key rotation.
- Handling undecryptable resources. Refer to the [KEP](https://github.com/kubernetes/enhancements/pull/3927) for details.

You can learn more about KMS v2 by reading [Using a KMS provider for data encryption](/docs/tasks/administer-cluster/kms-provider/). You can also follow along on the [KEP](https://github.com/kubernetes/enhancements/blob/master/keps/sig-auth/3299-kms-v2-improvements/#readme) to track progress across the coming Kubernetes releases.

## Call to action

## What’s next?
In this blog post, we have covered the improvements made to the KMS encryption provider in Kubernetes v1.27. We have also discussed the new KMS v2 API and how it works. We would love to hear your feedback on this feature. In particular, we would like feedback from Kubernetes KMS plugin implementors as they go through the process of building their integrations with this new API. Please reach out to us on the [#sig-auth-kms-dev](https://kubernetes.slack.com/archives/C03035EH4VB) channel on Kubernetes Slack.

## How to get involved
If you are interested in getting involved in the development of this feature, share feedback, or participate in any other ongoing SIG Auth projects, please reach out on the [#sig-auth](https://kubernetes.slack.com/archives/C0EN96KUY) channel on Kubernetes Slack.
Expand Down
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading

0 comments on commit 9dc2c6d

Please sign in to comment.