Gatekeeper MutatingAdmissionWebhook conflicts with vpc-cni

[tomerleib]

What steps did you take and what happened:
We have deployed Gatekeeper on our EKS cluster and created assign to inject a new sidecar for pods in one of the namespaces.
When a new pod that matched the assign was created, the sidecar was injected but VPC-CNI failed to assign an IP for this pod.

The error on the pod was Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "d456c3f7056f7574708efdcb161e676949989c33411c13eba09bb78502a9a7c3": plugin type="aws-cni" name="aws-cni" failed (add): add cmd: failed to assign an IP address to container.

After further investigation with AWS support, we discovered that there's a conflict between our vpc-cni that uses ANNOTATE_POD_IP and the gatekeeper webhook.

A similar issue can be found here: aws/amazon-vpc-cni-k8s#2654 (comment)

What did you expect to happen:
I was expecting the pod to start properly with network as everything else.

Anything else you would like to add:
[Miscellaneous information that will assist in solving the issue.]

Environment:

• Gatekeeper version: v3.10
• Kubernetes version: (use kubectl version): 1.30.6