AWS Provider Endpoints not working on 3.64.0

[iamperson347]

Terraform CLI and Terraform AWS Provider Version

Terraform v1.0.1
on linux_amd64

• provider registry.terraform.io/hashicorp/aws v3.64.0
• provider registry.terraform.io/hashicorp/vault v2.24.1

Relates #21641

Affected Resource(s)

AWS Provider

Terraform Configuration Files

Sample configuration for AWS Provider that is failing (some values replaced with <value>:

provider "aws" {
  alias  = "<alias name>"
  region = "us-gov-west-1"
  assume_role {
    role_arn     = "<role arn>"
    session_name = "<session name>"
  }

  endpoints {
    firehose = "https://firehose-fips.us-gov-west-1.amazonaws.com"
    s3       = "https://s3-fips.us-gov-west-1.amazonaws.com"
    lambda   = "https://lambda-fips.us-gov-west-1.amazonaws.com"
  }
}

Expected Behavior

terraform plan/apply works with custom AWS Service Endpoints

Actual Behavior

╷
│ Error: Unsupported argument
│
│ on providers.tf line 16, in provider "aws":
│ 16: firehose = "https://firehose-fips.us-gov-west-1.amazonaws.com"
│
│ An argument named "firehose" is not expected here.
╵
╷
│ Error: Unsupported argument
│
│ on providers.tf line 17, in provider "aws":
│ 17: s3 = "https://s3-fips.us-gov-west-1.amazonaws.com"
│
│ An argument named "s3" is not expected here.
╵
╷
│ Error: Unsupported argument
│
│ on providers.tf line 18, in provider "aws":
│ 18: lambda = "https://lambda-fips.us-gov-west-1.amazonaws.com"
│
│ An argument named "lambda" is not expected here.
╵

Steps to Reproduce

1. terraform apply

Important Factoids

References

• #0000

[nickcaballero]

Same issue when using Terraform JSON format.

{
  "provider": {
    "aws": [
      {
        "access_key": "test",
        "secret_key": "test",
        "region": "us-east-1",
        "endpoints": [
          {
            "cognitoidentity": "http://10.108.188.101:8890",
            "cognitoidp": "http://10.108.188.101:8890",
            "s3": "http://10.105.167.158:4566",
            "secretsmanager": "http://10.105.167.158:4566",
            "sns": "http://10.105.167.158:4566",
            "sqs": "http://10.105.167.158:4566"
          }
        ]
      }
    ]
  }
}

[perryao]

Found the issue.

:= on this line https://github.com/hashicorp/terraform-provider-aws/blob/main/internal/conns/conns.go#L580 is masking https://github.com/hashicorp/terraform-provider-aws/blob/main/internal/conns/conns.go#L577

[breathingdust]

Thanks all for issue report and help in narrowing it down the root cause, we plan to do a point release shortly to resolve this. 👍

[YakDriver]

@perryao Thank you for finding this. There is a test for custom endpoints (TestAccAcctestProvider_endpoints) but because the serviceData map was empty due to the issue you found, the test succeeded. Your fix plus additional repair will be in place shortly.