clusterctl move should pause all root hierachies

[fabriziopandini]

What would you like to be added (User Story)?

As an operator I would like to move my objects without controllers taking actions during the operation

Detailed Description

As of today clusterctl move pauses cluster and cluster class before moving so the corresponding controllers stops reconciling the objects being moved

The same should apply to all the other root hierarchies, like e.g. InClusterIPPool

Anything else you would like to add?

This issue has been discovered while investigating #8384

Label(s) to be applied

/kind feature
/area clusterctl

[srm09]

This would require us having to institute and update the contract so that IPPool implementations expose a paused field. The behavior of the implementation controllers should respect the value of this new field, similar to what the CAP* controllers do for the paused filed of the Cluster object.

The above seems like a good idea but is a departure from the current way of implementing infra specific objects and having corresponding objects in CAPI, read Cluster and VSphereCluster, or Machine and AWSMachine. It might make sense to augment the IPAM proposal to bring in a IPPool object which refers to a concrete/provider specific implementation. Something like,

apiVersion: cluster.x-k8s.io/v1beta1
kind: IPPool
metadata:
  name: system-pool
  namespace: default
spec:
  providerRef:
    apiVersion: ipam.cluster.x-k8s.io/v1alpha1
    kind: InClusterIPPool
    name: system-in-cluster-pool

This saves us from augmenting the contract and having to depend on the providers to implement this logic. It also means that we are following CAPI convention of keeping the provider specific logic separate and not being dependent on them. If we wanna also think about ClusterClass integration of IP pools, it'd follow the current convention and use an IPPoolTemplate object (although this one is not necessary a great advantage of this approach).

[srm09]

I am just proposing an alternate to the current approach, and would like to hear thoughts or suggestions from folks around this. @schrej @furkatgofurov7 wdyt?

[fabriziopandini]

We can both rely on a field or annotation, but before taking a decision we should take a deeper look at how pause filters works in CAPI, because they are dealing with most of the complexity here.

The above seems like a good idea but is a departure from the current way of implementing infra specific objects and having corresponding objects in CAPI

This is kind of necessary, because other root hierarchies like InClusterIPPool are not owned by any CAPI object.
The tricky part to figure out is how to define the behaviour of objects which are owned by two hierarchies, like the IPAddress object

TL;DR; +1 to hear more opinions and some more research work before taking next steps
/help

[k8s-ci-robot]

@fabriziopandini:
This request has been marked as needing help from a contributor.

Guidelines

Please ensure that the issue body includes answers to the following questions:

• Why are we solving this issue?
• To address this issue, are there any code changes? If there are code changes, what needs to be done in the code and what places can the assignee treat as reference points?
• Does this issue have zero to low barrier of entry?
• How can the assignee reach out to you for help?

For more details on the requirements of such an issue, please see here and ensure that they are met.

If this request no longer meets these requirements, the label can be removed
by commenting with the /remove-help command.

In response to this:

We can both rely on a field or annotation, but before taking a decision we should take a deeper look at how pause filters works in CAPI, because they are dealing with most of the complexity here.

The above seems like a good idea but is a departure from the current way of implementing infra specific objects and having corresponding objects in CAPI

This is kind of necessary, because other root hierarchies like InClusterIPPool are not owned by any CAPI object.
The tricky part to figure out is how to define the behaviour of objects which are owned by two hierarchies, like the IPAddress object

TL;DR; +1 to hear more opinions and some more research work before taking next steps
/help

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[fabriziopandini]

/triage accepted

[schrej]

This saves us from augmenting the contract and having to depend on the providers to implement this logic.

How? The providers will still reconcile the resources. If the label is set on some generic IPPool, it gets even more complex, since the provider needs to check that generic pool for a paused attribute or annotation whenever reconciling.

Machine exists because CAPI creates provider machines automatically and needs some way to track them. Provider clusters exist so provider configuration can be provided on a cluster level.
The only reason we could need an IPPool would be to store some generic IPAM related information, but it seems overkill to add that just to have some paused parameter there.

[k8s-triage-robot]

This issue has not been updated in over 1 year, and should be re-triaged.

You can:

• Confirm that this issue is still relevant with /triage accepted (org members only)
• Close this issue with /close

For more details on the triage process, see https://www.kubernetes.dev/docs/guide/issue-triage/

/remove-triage accepted