AWS S3 state backend fails with AWS SSO new profile format

[chiemerieezechukwu]

Terraform Version

Terraform v1.3.7
on darwin_arm64
+ provider registry.terraform.io/datadog/datadog v3.19.1
+ provider registry.terraform.io/hashicorp/aws v3.75.2
+ provider registry.terraform.io/hashicorp/null v3.2.1

Terraform Configuration Files

terraform {
  required_version = "~>1.0"

  required_providers {
    aws = {
      version = "~>3.0"
      source  = "hashicorp/aws"
    }
    null = {
      version = "~>3.0"
      source  = "hashicorp/null"
    }
    datadog = {
      source  = "DataDog/datadog"
      version = "~> 3.18"
    }
  }

  backend "s3" {
    region         = "eu-central-1"
    bucket         = "xxxx-bucket-xxxx"
    key            = "state-key"
    dynamodb_table = "lock-table"
    encrypt        = "true"
  }
}

Debug Output

...
2023-01-05T17:27:57.944+0100 [DEBUG] ignoring non-existing provider search directory /Library/Application Support/io.terraform/plugins
2023-01-05T17:27:57.945+0100 [INFO]  CLI command args: []string{"apply"}
2023-01-05T17:27:57.954+0100 [TRACE] Meta.Backend: built configuration for "s3" backend with hash value 3591387704
2023-01-05T17:27:57.955+0100 [TRACE] Preserving existing state lineage "8e30792b-97ac-5674-d17a-955b1985e845"
2023-01-05T17:27:57.955+0100 [TRACE] Preserving existing state lineage "8e30792b-97ac-5674-d17a-955b1985e845"
2023-01-05T17:27:57.956+0100 [TRACE] Meta.Backend: working directory was previously initialized for "s3" backend
2023-01-05T17:27:57.956+0100 [TRACE] Meta.Backend: using already-initialized, unchanged "s3" backend configuration
2023-01-05T17:27:57.959+0100 [INFO]  Attempting to use session-derived credentials
╷
│ Error: error configuring S3 Backend: Error creating AWS session: profile "xxx" is configured to use SSO but is missing required configuration: sso_region, sso_start_url
│ 
│ 
╵

Expected Behavior

Terraform should work with the new AWS config format

Actual Behavior

Terraform throws error with the new AWS config format

Steps to Reproduce

1. Configure S3 remote state backend (including creating the bucket, making sure your AWS permissions are sufficient, etc.)
2. Configure your AWS client to use AWS IAM Identity Center (SSO) new format. Example ~/.aws/config entry:

[profile xxx]
sso_session = my-sso
sso_account_id = 111122223333
sso_role_name = SampleRole

[sso-session my-sso]
sso_region = us-east-1
sso_start_url = https://my-sso-portal.awsapps.com/start
sso_registration_scopes = sso:account:access

reference: see here

3. Login with AWS_PROFILE=xxx aws sso login
4. confirm that your bucket is accessible (e.g. AWS_PROFILE=xxx AWS_REGION=yyy aws s3 ls <your-bucket-name>)
5. Run e.g. terraform init, and receive the error message above.

Additional Context

No response

References

No response

[crw]

Thanks for the report, I'll notify the AWS provider team.

[simon97k]

Any updates on this?

[jlarfors]

In the meantime, a workaround I found is to use the legacy non-refreshable configuration as per the AWS docs: https://docs.aws.amazon.com/cli/latest/userguide/cli-configure-sso.html#sso-configure-profile-legacy

I created my profile manually according to the above docs, which now contains the sso_region and sso_start_url fields, and ran aws sso login --profile <legacy-profile-name> and can now use my S3 backend.

Hopefully this gets fixed soon!

[jparta]

An issue exists since early December: hashicorp/terraform-provider-aws#28263

[tonyhhyip]

The sso_session is not supported by AWS Go SDK v1, which is used by Terraform. To fix this issue, it would require migrating to v2 SDK.

[omerfsen]

https://docs.aws.amazon.com/cli/latest/userguide/cli-configure-sso.html#sso-configure-profile-legacy

I manually export

and it works.. Though I need to do it every 8 hours ;) manually.

[omerfsen]

hashicorp/terraform-provider-aws#28263 (comment) provides a workaround with S3 backend for now..

[pjaudiomv]

hashicorp/terraform-provider-aws#28263 (comment) provides a workaround with S3 backend for now..

yea doesn't help when you use many aws accounts though or a different aws account for backend vs provider :(. you have to login to every account separately vs just one with session chaining.