[WIP] Tide-up helm chart

[mjudeikis]

Currently Helm chart is not working with community version. There are a lot of values missing and virtual workspaces do no work outside process.

This PR is not working too as yet. Its raised more as visibility that I'm working on refactoring it a bit so if somebody wants to pick-up/reuse/help - feel free. Once I get to the place where it deploys confidently and I will clien commits and prepare to merge

For now its blocked by:
kcp-dev/kcp#2254
kcp-dev/kcp#2295
kcp-dev/kcp#2325
kcp-dev/kcp#2184

TODO:

• Make use of community images
• make VW running in separate process
• Make Shards scalable (Deployment to scaleSet? )
• Generate development/test kubeconfig for auth-proxy

[mjudeikis]

dev image

[yogesh-reddy]

@mjudeikis I have tried the helm chart with your changes in a kind cluster.
and I see the pods kcp and kcp-front-proxy do not come up
this is what I get when I describe the pod
Normal Scheduled 5m25s default-scheduler Successfully assigned default/kcp-5f96575784-7cxdd to kcp-cluster-worker2 Warning FailedMount 5m10s (x6 over 5m25s) kubelet MountVolume.SetUp failed for volume "etcd-certs" : secret "kcp-etcd-client" not found Warning FailedMount 5m10s (x6 over 5m25s) kubelet MountVolume.SetUp failed for volume "kcp-requestheader-client-ca" : secret "kcp-requestheader-client-ca" not found Warning FailedMount 5m10s (x6 over 5m25s) kubelet MountVolume.SetUp failed for volume "kcp-server-client-ca" : secret "kcp-server-client-ca" not found Warning FailedMount 4m54s (x7 over 5m25s) kubelet MountVolume.SetUp failed for volume "kcp-certs" : secret "kcp-cert" not found Warning BackOff 24s (x18 over 2m30s) kubelet Back-off restarting failed container

i have tried installing the cert-manager on the same cluster .
used jetstack certmanager helm chart for this
helm install \ cert-manager jetstack/cert-manager \ --namespace cert-manager \ --create-namespace \ --version v1.10.1 \

let me know if i am missing anything.

thanks in advance

[mjudeikis]

Chart is pointing to main branch images and is not up-to-date after refactoring changes merged recently. Need more work to adjust to new workspace structure :/

So I 100% expect it to be broken now :(

[yogesh-reddy]

@mjudeikis so i was working on deploying the kcp workload on k8s cluster .. is this helm chart the only way or do we have some other ways to achieve that ?

[mjudeikis]

There multiple aspects to this. Let me respond In more details tomorrow. Not at pc now :)

Update: As promised. KCP is inteded to be running like this:
FrontProxy ---> N x shards

So deployment via helm in production is complicated due to fact shards needs to share assets, like certificates. So current helm chart (assuming we can refactor changes to represent what is in main) would be able to deploy potentially multiple shards inside the same k8s undelaying cluster. Having said this, this does not quite represent real production usecase. I know @vincepri was looking to some documentation how to run this, and I was bit waiting for it to to complete this PR.

If you want to help me, this is best pointer for how to run KCP in the multi-shard, single proxy setup https://github.com/kcp-dev/kcp/tree/main/cmd/sharded-test-server
Other that that, helm method of deployment still has a downside when assets needs to be shared via multiple cluster so potentially some hacking or other tool will need to be created.

[yogesh-reddy]

@mjudeikis I may not need production quality with sharding now .. I can just live with one instance.
my use case is as simple as running minimal kcp like how we run with binaries locally.
so is this possible .. if not by helm charts at least by some bunch of deployment YAML?

[mjudeikis]

I dont have those. And as to my knowledge there is nothing public yet. As said for time being https://github.com/kcp-dev/kcp/tree/main/cmd/sharded-test-server is best referece how to do so.