When setup-gcp bootstrap runs against an existing cluster that has GKE's managed Filestore CSI driver enabled, its reconcile path turns the driver off (cluster.go#L274-L297). It does this unconditionally, with no prompt, as a control-plane update of about ten minutes.
Disabling it on a cluster setup-gcp creates makes sense. #1463 did that because Substrate needs its own build of the driver ("A custom version is required for now"), and a new cluster has nothing depending on the managed one. An existing cluster is different:
- Workloads that already use the managed driver lose it. For example, PVCs on GKE's built-in RWX Filestore storage classes. I'd expect new volume provisioning, and new pods mounting existing Filestore volumes, to fail until another driver is running. Mounts already in place are probably unaffected. (I haven't tested the workload impact directly; I did see the reconcile disable the add-on on a test cluster:
Mismatch in Filestore CSI driver config current=true expected=false.)
- Nothing replaces it at that point. Bootstrap disables the managed driver but doesn't deploy Substrate's build; that's a separate, optional step. A user who doesn't plan to use Filestore with Substrate loses the cluster's Filestore support for nothing.
- It's another ~10-minute control-plane update on top of the others the reconcile makes (Workload Identity, beta APIs, managed OpenTelemetry).
Proposal: leave an existing cluster's Filestore add-on alone in the reconcile path, and turn it off where Substrate's driver is actually deployed. That's where the GKE installer already does it: its Filestore step checks addonsConfig.gcpFilestoreCsiDriverConfig.enabled and disables the add-on right before deploying the Substrate overlay. Keep the disable in buildCreateClusterRequest for clusters setup-gcp creates.
Related: #2341, the same reconcile path's delete-and-recreate on a network mismatch. Happy to send the PR.
When
setup-gcp bootstrapruns against an existing cluster that has GKE's managed Filestore CSI driver enabled, its reconcile path turns the driver off (cluster.go#L274-L297). It does this unconditionally, with no prompt, as a control-plane update of about ten minutes.Disabling it on a cluster setup-gcp creates makes sense. #1463 did that because Substrate needs its own build of the driver ("A custom version is required for now"), and a new cluster has nothing depending on the managed one. An existing cluster is different:
Mismatch in Filestore CSI driver config current=true expected=false.)Proposal: leave an existing cluster's Filestore add-on alone in the reconcile path, and turn it off where Substrate's driver is actually deployed. That's where the GKE installer already does it: its Filestore step checks
addonsConfig.gcpFilestoreCsiDriverConfig.enabledand disables the add-on right before deploying the Substrate overlay. Keep the disable inbuildCreateClusterRequestfor clusters setup-gcp creates.Related: #2341, the same reconcile path's delete-and-recreate on a network mismatch. Happy to send the PR.