Kind Operator

[shaneutt]

What would you like to be added:

A Kubernetes operator capable of provisioning and maintaining the lifecycle off multiple Kind clusters.

Clusters would be run in containers in any existing Kubernetes clusters and would enable:

• configuration
• deployment
• runtime operations
• automated cleanup

For any number of Kind clusters.

Why is this needed:

Currently kind makes it straightforward to quickly deploy a Kubernetes cluster for integration tests and ultimately be thrown away.

It does however tend to be the case that these workflows are done via command line and using local system resources and container runtimes.

This also means that the time cost to build and tear down the cluster is often paid for by the CI where the tests are run, and problems with provisioning the cluster are problems the integration tests unrelated to provisioning inherit.

This feature would enable hosting Kind as a service such that CI infrastructure could separate the concerns of creating and maintaining the Kind containers. This would also present some speed benefits as a number of clusters could be proactively created to keep a "ready pool" of fresh clusters wherever a CI run claims an existing one.

This would enable test runs to create and/or "claim" clusters using a Kubernetes client and access to the host cluster rather than needing a local systems resources and container runtime.

[BenTheElder]

See #1303 We do NOT recommend running Kubernetes in Kubernetes. There are a lot of problems.

If you're going to run clusters as a service, why not use any existing cluster as a service options?

As for a ready pool operator: this is https://github.com/kubernetes-sigs/boskos/tree/master/mason

This sounds extremely out of scope.

EDIT: "hit close and comment" on accident while still typing.

[BenTheElder]

Also short of docker caching concerns (which are environment specific) kind aims to create clusters in under a minute, I'd rather see us improve cluster startup cost than ship a likely non-portable operator that also encourages the CI workflow to be different versus the local developer workflow. Being able to throw away the cluster and test from a clean state is a core goal of the project.

[BenTheElder]

Actually, I think https://github.com/kubernetes-sigs/cluster-api/tree/master/test/infrastructure/docker is fairly close to what you want.

An operator managing cluster lifecycle is basically describing Cluster-API, loosely, and they have an implementation of that API based on KIND for the purpose of testing Cluster-API.

[shaneutt]

Thanks for your considerations. I am familiar with CAPI so I'll consider retargeting my interest in that direction as per your suggestion.

[BenTheElder]

Thanks for understanding and thanks for bringing this up in detail for consideration 😅