Kube2iam not assuming roles on kops

[craig-dsilva]

I have Kubernetes (v 1.28.4) running with Kops (v 1.28.1 previously running v1.19) on AWS. Recently we upgraded the cluster and kube2iam stopped working and is not assigning roles to the pods. I don't know what has gone wrong as it used to work with the previous version. Even the logs do not give me too much info.

[elliotdobson]

What version of kube2iam are you using?

As noted in the kOps v1.27 release notes:

As of Kubernetes version 1.27, all nodes will default to running with instance-metadata-service tokens required, with a max hop limit of 1. Newly created clusters will be configured as necessary to have these settings.

Which means that IMDSv2 is required in kOps Kubernetes v1.27+ clusters.

You can either:

• Enable IMDSv1 as described here.
  OR
• Upgrade to kube2iam v0.11.2 which (IIUC) supports IMDSv2.

[act-mreeves]

Sadly, nope. See #376.

Comparing 0.11.2 (latest release) to master shows the imdsV2 PR has not been added to a release: 0.11.2...master

[elliotdobson]

@act-mreeves the GitHub tags are misleading. There is discussion about misleading releases in #366 & #367.

Comparing 0.11.1...0.11.2 tags does not show the IMDSv2 feature (latest commit is 20/11/2023), however comparing 0.11.2...release-0.11.2 (tag/branch) it shows the IMDSv2 feature (latest commit is 27/11/2023).

The 0.11.2 docker image tag was last pushed on 27/11/2023. So IMDSv2 support is included in kube2iam v0.11.2 and I tested that to confirm.

We used the docker image as linked above to support IMDSv2 via kube2iam on a kOps v1.27+ cluster, which we have since migrated to IAM Roles for Service Accounts (IRSA).