Using sysbox with autoscaling nodes: deadlock

[victornoel]

Hi,

I'm trying to use sysbox (as a daemon set) in GKE cluser, and I'm using autoscaler to create nodes only if there is a pod to be created.

The problem is that the pod is created, and because it is scheduled, then the node is created, and the following happens:

• the pod is created
• sysbox-deploy-k8s is started to install sysbox stuff and then it kills every pod including the pod which triggered the node creation

I see that there is a taint used to prevent pod deployments before sysbox is done sysbox-runtime=not-running:NoSchedule, but it's already too late because the pod is already scheduled to be able to create the node (if I understood correctly).

Are there known way to solve this elegantly?

Thanks!