Disable GitOps behavior on Azure is inconsistent with flux uninstall

[kingdonb]

Expected behaviour

When the Disable GitOps command is invoked, the generic Flux provider basically just invokes flux uninstall

This is a very safe way to disable GitOps as it leaves all the workloads intact. The Flux controllers are removed, then the flux CRDs are removed, and any finalizers that blocked Flux resources from being deleted are manually cleaned up by the uninstall routine, when the flux-system namespace is then also (optionally) removed. This means that prune or Garbage Collection behavior is never invoked during this operation.

When the Disable GitOps command is invoked on Azure providers, I'm afraid that is not what happens:

Actual behaviour

I'm going to make some assumptions because I am not completely up to speed on how every bit of the Azure Flux extension works, but here goes (from src/azure/azureTools.ts)

const fluxConfigurations = await this.listFluxConfigurations(contextName, clusterProvider);

if (!fluxConfigurations) {
  return;
}

const namesOfFluxConfigs: string[] = fluxConfigurations.map((configuration: {name: string;}) => configuration.name);

// delete all flux configurations
await Promise.all(namesOfFluxConfigs.map(fluxConfigName => this.invokeAzCommand(
  `az k8s-configuration flux delete -n ${fluxConfigName} --yes`,
  contextName,
  clusterProvider,
)));

// delete flux extension
const disableGitOpsShellResult = await this.invokeAzCommand(
  `az k8s-extension delete --name ${AzureConstants.FluxExtensionName} --yes`,
  contextName,
  clusterProvider,
);
if (disableGitOpsShellResult?.code !== 0) {
  telemetry.sendError(TelemetryErrorEventNames.FAILED_TO_RUN_AZ_DISABLE_GITOPS);
}

If we enumerate the fluxConfigurations then use az k8s-configuration flux delete ... on each one, if any had prune set then their workloads will be deleted. This is inconsistent with flux uninstall – but – it might also be desirable, as it might be the expectation that you "Disable GitOps" at the end of your experiment and things are cleaned up after that.

My preference is that the behavior is comparable across all providers. If generic Flux uninstall cleans up CRDs but leaves the workloads behind, then uninstalling on Azure should work the same way. If you want Flux to clean up the workloads then they must be deleted from Git, or deleted with garbage collection by the workloads, before the uninstall operation is engaged.

Steps to reproduce

I do not have steps to reproduce, but Disable GitOps should trigger the behavior.

I am going by a reading of the code, I do not have an Azure system handy for testing yet. (Planned for next week...)

We should merge #339 before we fix this, so the users can delete workloads manually before they disable GitOps. Otherwise there will be a new problem as cleaning up after an experiment will no longer be possible from the UI via either provider.

Versions

kubectl version:
Flux version:
Git version:
Azure version:
Extension version: v0.20.8, v0.20.9, earlier versions
VSCode version:
Operating System (OS) and its version:

[kingdonb]

@juozasg just raising this to your attention, I'm not sure if there's anything actionable in here now.

(Please take a look at the 0.25 milestone, I'm moving things there that I think we should address before another MINOR release. 👍 everything else if it looks important but not urgent is going to go into 0.26)