Support for AWS access via IAM - AssumeRole

[ncraike]

I get it's probably a lot of complexity to add, but it'd be great if Terraform supported using IAM roles in its AWS provider authentication.

This adds a few steps to how API credentials are used – you use your normal AWS access key and secret key to call sts assume-role in the API, and this responds with a temporary access key, secret key and session token which you use for the rest of the session. An example with AWS's own CLI is here.

The big use case for this is cross-account access. We have lots of AWS accounts to isolate billing to individual projects, and we link these to one parent account with AWS's consolidated billing. The cross-account roles let us have one set of IAM users in the parent account (eg one IAM user for each actual developer using AWS), and then just a single IAM role in each sub-account, which the developers can assume when needed. This saves having to create a new set of IAM users in each sub-account.

I imagine this would involve adding extra AWS provider config fields to describe the role to assume (using an AWS ARN like arn:aws:iam::account-of-role-to-assume:role/name-of-role), and would also involve changing the AWS API access layer to use the temporary role access/secret keys and a session token.

This should be linked to the AWS meta-issue. I felt the explanation was long enough to warrant a full issue.

[radeksimko]

I'm currently working on #1049 which is providing IAM support via the aws-sdk-go library, but it's not covering the assume-role use-case (yet?).

The only use-case for the IAM authorisation at the moment is to call the metadata API from inside a running EC2 instance: https://github.com/awslabs/aws-sdk-go/blob/440f9da54cf714e7c27b5b4ffa789793a5fbd658/aws/auth.go#L203-228

To fully understand the suggested solution, would you expect the provider config to have following options?

provider "aws" {
  credentials_provider = "iam"
  iam_role_arn = "arn:aws:iam::account-of-role-to-assume:role/name-of-role"
}

How do you expect to provide the initial pair of access_key & secret_key to call the STS API which will give you the token for further API requests? Statically? File config? ENV vars?

Related: http://docs.aws.amazon.com/IAM/latest/UserGuide/roles-usingrole-switchapi.html

[radeksimko]

Also if you use MFA for the AssumeRole, it would require some other changes in the whole TF CLI workflow as well, e.g. Terraform interactively asking for the TOTP and providing it in that API request to STS.

[radeksimko]

btw. there's a lot more use-cases = workflows for this...

http://docs.aws.amazon.com/cli/latest/reference/sts/assume-role-with-web-identity.html
http://docs.aws.amazon.com/cli/latest/reference/sts/assume-role-with-saml.html

Ain't saying it should all be covered in the first implementation, just thinking loudly about it...

[ncraike]

Adding an iam_role_arn config is about as much as I think would be required, though maybe iam_role_to_assume or similar might be more meaningful and readable.

How do you expect to provide the initial pair of access_key & secret_key to call the STS API which will give you the token for further API requests? Statically? File config? ENV vars?

I would expect to provide them the same way access_key and secret_key are currently provided, as fields within the provider "aws" { block, as in the current docs.

I'm not sure if the credentials_provider = "iam" setting is needed, unless we're going to add a lot more providers.

[ncraike]

So, to clarify, I'd expect this to be enough for the cross-account assume-role use case:

provider "aws" {
    access_key = "${var.aws_access_key}"
    secret_key = "${var.aws_secret_key}"
    iam_role_to_assume = "arn:aws:iam::account-number:role/name-of-role"

    region = "us-east-1"
}

[ncraike]

I admit the other role workflows and MFA would add a lot of scope.

I'm primarily interested in this for the cross-account use case I described, because it seems to be the only method AWS supports for accessing a consolidated billing sub-account from the parent account, other than creating a new set of IAM users for each sub-account.

[ncraike]

Ah, looking at PR #1049, it does look like you've added support for more complex AWS credential provider configurations?

Sorry, I don't actually know the Terraform codebase or goamz, aws-go-sdk, etc very well. Currently I don't know Go (though Terraform may motivate me to learn), so my issues and comments are largely from the point-of-view of someone using Terraform and using AWS's own tools and API.

[willmcg]

Another use case for IAM roles is when you are running Terraform from an EC2 instance with an IAM role in its instance profile. In this case the temporary access credentials (access, secret & token) for the account can be obtained directly from the EC2 instance metadata. This relieves you from the need to manage these credentials manually if you are willing to run terraform in an EC2 instance and would make things very easy for the use case of deploying into the same account.

For the cross-account access you would only need to specify the IAM role to assume in the other account and Terraform would call AssumeRole using the credentials from the instance profile to get a new set of access credentials for the other account.

These temporary credentials from the instance profile can expire so some logic would need to be implemented to renew from the instance metadata when necessary, although it is unlikely that a terraform invocation would run longer than the minimum lifetime a set of credentials obtained from the instance metadata.

[radeksimko]

@willmcg

Another use case for IAM roles is when you are running Terraform from an EC2 instance with an IAM role in its instance profile.

Yes, that's already included in the linked #1049

For the cross-account access you would only need to specify the IAM role to assume in the other account and Terraform would call AssumeRole using the credentials from the instance profile to get a new set of access credentials for the other account.

Good point with cross-account access in general. I'd personally say that it's actually a reason to implement the whole assume-role logic separately in another PR as it's increasing the scope of the original PR quite significantly.

These temporary credentials from the instance profile can expire so some logic would need to be implemented to renew from the instance metadata when necessary.

The renewing logic is already part of the go library we use:
https://github.com/awslabs/aws-sdk-go/blob/440f9da54cf714e7c27b5b4ffa789793a5fbd658/aws/auth.go#L212-217

The expiration period though is unfortunately currently hardcoded. I will have a look if there's any way we can find the expiration period automatically, if not, I will probably expose it as another variable, that user will need to set or overwrite.

[willmcg]

Ah... thanks for the code pointers. Agreed on splitting to another PR... significant scope creep :-)

The expiration time is available in the EC2 metadata along with the temporary credentials:

http://169.254.169.254/latest/meta-data/iam/security-credentials/ROLE/Expiration

[radeksimko]

The expiration time is available in the EC2 metadata along with the temporary credentials:

👓 read the code more carefully, before you post, Radek... 😑

My bad, the hard-coded time period actually only applies to token provided from a config file:
https://github.com/awslabs/aws-sdk-go/blob/master/aws/auth.go#L130
there might be a way on how to track down the IAM policy which has the time period value, but it may not be that easy as the user calling the API may not have access to the actual IAM resource (can't see any details of policies which are affecting him).
More importantly even if we track it down, then we've got another much bigger problem ahead. These temporary credentials inside files are usually generated using another set of credentials (which only have privilege to generate tokens and not access any other APIs) and the generation process often includes MFA/SAML authorisation.

The IAM provider already reads the expiration period from the API:
https://github.com/awslabs/aws-sdk-go/blob/master/aws/auth.go#L250-259

All of the above means that for the basic implementation we're IMO good to go with #1049 😃

[radeksimko]

@ncraike would you mind modifying the issue title to "Support for AWS access via IAM - AssumeRole" or something along those lines?

[ncraike]

Done.

I feel I've opened a can of worms here, though.