After clusterctl move, vsphere worker machines take ~10-12 minutes to go to "Running" phase

[mrajashree]

/kind bug

What steps did you take and what happened:
Created a self-managed capv cluster following steps in the quickstart book

1. Created a Kind bootstrap cluster
2. Installed CAPI+CAPV providers through clusterctl
3. Created the target workload cluster
4. Installed CAPI+CAPV providers through clusterctl on this workload cluster
5. Performed clusterctl move to the workload cluster.

What did you expect to happen:
The worker machines' status field has the nodeRef value set after move, because of which it can be considered ready, but its phase is Provisioned and not Ready for about 10-12 minutes after the move.

Anything else you would like to add:
These are the Machine.Status fields of a worker machine post move, the difference between the NodeHealthy and Ready conditions' timestamps is ~13 min, which is the time taken for Machine to go to "Running" phase.

status:
  addresses:
  - address: 198.x.x.x
    type: ExternalIP
  bootstrapReady: true
  conditions:
  - lastTransitionTime: "2021-07-13T23:13:47Z"
    status: "True"
    type: Ready
  - lastTransitionTime: "2021-07-13T22:55:52Z"
    status: "True"
    type: BootstrapReady
  - lastTransitionTime: "2021-07-13T23:13:47Z"
    status: "True"
    type: InfrastructureReady
  - lastTransitionTime: "2021-07-13T22:55:52Z"
    status: "True"
    type: NodeHealthy
  infrastructureReady: true
  lastUpdated: "2021-07-13T23:13:47Z"
  nodeRef:
    apiVersion: v1
    kind: Node
    name: abcd
    uid: 2dc6356a-362b-432e-b573-026d2193f17a
  observedGeneration: 1
  phase: Running

And after move before those 13 minutes finish, the Ready condition is set to False with WaitingForClusterInfrastructure and none of the addresses are set.

Environment

• Cluster-api-provider-vsphere version: v0.7.8
• Kubernetes version: (use kubectl version): 1.19
• OS (e.g. from /etc/os-release): Ubuntu 18.04

[mrajashree]

/assign @mrajashree

[k8s-triage-robot]

The Kubernetes project currently lacks enough contributors to adequately respond to all issues and PRs.

This bot triages issues and PRs according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Mark this issue or PR as fresh with /remove-lifecycle stale
• Mark this issue or PR as rotten with /lifecycle rotten
• Close this issue or PR with /close
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale