Fails on EC2 instances with multiple interfaces

[yannh]

On a Kubernetes cluster setup with KOPS & aws-vpc-k8s-cni, k8s-ec2-srcdst fails to disable srcdst with the following message:
"srcdst_controller.go:87] Fail to disable src dst check for EC2 instance: i-xxxxx; InvalidInstanceID: There are multiple interfaces attached to instance 'i-xxxxxx'. Please specify an interface ID for the operation instead."

I believe this is because aws-vpc-k8s-cni creates instances with several network interfaces - and srcdst needs to be disabled on each of them separately. The following documentation mentions a different procedure for an instance with more than a single interface: https://docs.aws.amazon.com/vpc/latest/userguide/VPC_NAT_Instance.html#EIP_Disable_SrcDestCheck

Maybe k8s-ec2-srcdst should list the interfaces for the current instances and disable srcdestcheck for each of them?

[ottoyiu]

This is related to #7. I'm not very familar with the CNI driver... but why does the aws-vpc-k8s-cni driver require disabling srcdst checks on your EC2 instances? My assumption was that it uses real VPC routable IP addresses that are bound to an ENI so that you can avoid passing traffic around using IP addresses that AWS is not aware of.