WIP: cluster-api - boot nodes without state store access #8198
Conversation
We were previously building it in development using bazel, but then using go build for the release. That is risky, better to just use bazel everywhere.
We want to start re-using it for cluster-api support.
This is the first step to supporting cluster-api; starting with GCE for now. Slight problem at the moment: kubernetes/kubernetes#86509
This allows us to create an InstanceGroup without needing to upload to the state store (S3/GCS)
If we're not going to use the docker images, we don't need to preload them - saves a bit off the boot time.
k8s-ci-robot
added
do-not-merge/work-in-progress
|
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: justinsb The full list of commands accepted by this bot can be found here. The pull request process is described here
Needs approval from an approver in each of these files:
Approvers can indicate their approval by writing |
k8s-ci-robot
added
the
approved
| Contents: fi.NewBytesResource(pkiutil.EncodePrivateKeyPEM(serverKey)), | ||
| Type: nodetasks.FileType_File, | ||
|
|
||
| // TODO: Because we run as an unprivileged user (1000), we can't use hostPaths with mode 600 |
There was a problem hiding this comment.
Perhaps specify the file Owner?
| } | ||
|
|
||
| func (b *Builder) BuildMachineDeployment(clusterObj *kopsv1alpha2.Cluster, igObj *kopsv1alpha2.InstanceGroup) ([]*unstructured.Unstructured, error) { | ||
| cloudup.AlphaAllowGCE.SetEnabled(true) |
There was a problem hiding this comment.
I guess it is necessary to enable this builder to run GCE. But I wonder if we can move it somewhere else, where it does not bloat a unrelated context.
k8s-ci-robot
added
the
needs-rebase
|
@justinsb: PR needs rebase. Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. |
|
@justinsb: The following tests failed, say
Full PR test history. Your PR dashboard. Please help us cut down on flakes by linking to an open issue when you hit one in your PR. Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here. |
|
This one is partially superseded by #10469, but this one also adds support for MachineDeployment resources |
|
Issues go stale after 90d of inactivity. If this issue is safe to close now please do so with Send feedback to sig-contributor-experience at kubernetes/community. |
k8s-ci-robot
added
the
lifecycle/stale
|
Stale issues rot after 30d of inactivity. If this issue is safe to close now please do so with Send feedback to sig-contributor-experience at kubernetes/community. |
k8s-ci-robot
added
lifecycle/rotten
|
Rotten issues close after 30d of inactivity. Send feedback to sig-contributor-experience at kubernetes/community. |
|
@k8s-triage-robot: Closed this PR. In response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. |
This is the other half of the initial cluster-api spike (the first half is #8151).
Instead of requiring nodes to read from the state store (s3 or GCS) in nodeup, we spin up a GRPC server in kops-controller that lets us serve the artifacts. Currently they're still loaded from S3/GCS by kops-controller, but this means the node doesn't need permissions. We're also getting closer to no S3/GCS dependency, if kops-controller can generate more of these artifacts on the fly (it's only the kubelet & kube-proxy keypairs).
What this does now allow is dynamic creation of InstanceGroups via the k8s api.