[Bug]: aws_api_gateway_method_settings - reading API Gateway Method Settings empty results

[jkoermer-eqxm]

Terraform Core Version

1.8.5

AWS Provider Version

5.53.0

Affected Resource(s)

aws_api_gateway_method_settings
aws_api_gateway_rest_api

Expected Behavior

The terraform plan shows the following:

  # module.api_gateway.aws_api_gateway_method_settings.endpoint_throttle will be created
  + resource "aws_api_gateway_method_settings" "endpoint_throttle" {
      + id          = (known after apply)
      + method_path = "path/{proxy+}/GET"
      + rest_api_id = "xxxxxxxxxx"
      + stage_name  = "develop"

      + settings {
          + cache_data_encrypted                       = (known after apply)
          + cache_ttl_in_seconds                       = (known after apply)
          + caching_enabled                            = (known after apply)
          + data_trace_enabled                         = (known after apply)
          + logging_level                              = (known after apply)
          + metrics_enabled                            = true
          + require_authorization_for_cache_control    = (known after apply)
          + throttling_burst_limit                     = 80
          + throttling_rate_limit                      = 200
          + unauthorized_cache_control_header_strategy = (known after apply)
        }
    }

We should see the settings applied to that path in the associated stage for the api gateway.

Actual Behavior

�[31m╷�[0m�[0m
�[31m│�[0m �[0m�[1m�[31mError: �[0m�[0m�[1mreading API Gateway Method Settings (<gatewayid>-<env>-path/{proxy+}/GET): empty result�[0m
�[31m│�[0m �[0m
�[31m│�[0m �[0m�[0m  with module.api_gateway.aws_api_gateway_method_settings.endpoint_throttle["path_get"],
�[31m│�[0m �[0m  on ../../modules/aws/api_gateway/main.tf line 76, in resource "aws_api_gateway_method_settings" "endpoint_throttle":
�[31m│�[0m �[0m  76: resource "aws_api_gateway_method_settings" "endpoint_throttle" �[4m{�[0m�[0m
�[31m│�[0m �[0m
�[31m╵�[0m�[0m

Relevant Error/Panic Output Snippet

No response

Terraform Configuration Files

data "aws_api_gateway_rest_api" "api_gateway" {
  name = "${var.environment}-api"
}

resource "aws_api_gateway_method_settings" "endpoint_throttle" {
  rest_api_id = data.aws_api_gateway_rest_api.api_gateway.id
  stage_name  = var.environment
  method_path = "gateway/path/{proxy+}/GET"
  settings {
    metrics_enabled        = true
    throttling_burst_limit = 80
    throttling_rate_limit  = 200
   }
}

Steps to Reproduce

terraform plan shows:

  # module.api_gateway.aws_api_gateway_method_settings.endpoint_throttle will be created
  + resource "aws_api_gateway_method_settings" "endpoint_throttle" {
      + id          = (known after apply)
      + method_path = "gateway/path/{proxy+}/GET"
      + rest_api_id = "xxxxxxxxxx"
      + stage_name  = "develop"

      + settings {
          + cache_data_encrypted                       = (known after apply)
          + cache_ttl_in_seconds                       = (known after apply)
          + caching_enabled                            = (known after apply)
          + data_trace_enabled                         = (known after apply)
          + logging_level                              = (known after apply)
          + metrics_enabled                            = true
          + require_authorization_for_cache_control    = (known after apply)
          + throttling_burst_limit                     = 80
          + throttling_rate_limit                      = 200
          + unauthorized_cache_control_header_strategy = (known after apply)
        }
    }

apply fails with the following error:

Error: reading API Gateway Method Settings (xxxxxxxx-develop-pgateway/path/{proxy+}/GET): empty result with module.api_gateway.aws_api_gateway_method_settings.endpoint_throttle,
on ../../modules/aws/api_gateway/main.tf line 76, in resource "aws_api_gateway_method_settings" "endpoint_throttle":
76: resource "aws_api_gateway_method_settings" "endpoint_throttle"

Debug Output

This is part of a much larger workspace, from what I can tell this is the specifics about the method settings:

2024-06-13T18:06:03.712Z [TRACE] maybeTainted: module.api_gateway.aws_api_gateway_method_settings.endpoint_throttle["path_get"] encountered an error during creation, so it is now marked as tainted
2024-06-13T18:06:03.712Z [TRACE] terraform.contextPlugins: Schema for provider "registry.terraform.io/hashicorp/aws" is in the global cache
2024-06-13T18:06:03.712Z [TRACE] NodeAbstractResouceInstance.writeResourceInstanceState to workingState for module.api_gateway.aws_api_gateway_method_settings.endpoint_throttle["path_get"]
2024-06-13T18:06:03.712Z [TRACE] NodeAbstractResouceInstance.writeResourceInstanceState: writing state object for module.api_gateway.aws_api_gateway_method_settings.endpoint_throttle["path_get"]
2024-06-13T18:06:03.712Z [TRACE] evalApplyProvisioners: module.api_gateway.aws_api_gateway_method_settings.endpoint_throttle["path_get"] is tainted, so skipping provisioning
2024-06-13T18:06:03.712Z [TRACE] maybeTainted: module.api_gateway.aws_api_gateway_method_settings.endpoint_throttle["path_get"] was already tainted, so nothing to do
2024-06-13T18:06:03.712Z [TRACE] terraform.contextPlugins: Schema for provider "registry.terraform.io/hashicorp/aws" is in the global cache
2024-06-13T18:06:03.712Z [TRACE] NodeAbstractResouceInstance.writeResourceInstanceState to workingState for module.api_gateway.aws_api_gateway_method_settings.endpoint_throttle["path_get"]
2024-06-13T18:06:03.712Z [TRACE] NodeAbstractResouceInstance.writeResourceInstanceState: writing state object for module.api_gateway.aws_api_gateway_method_settings.endpoint_throttle["path_get"]
2024-06-13T18:06:03.715Z [DEBUG] State storage *cloud.State declined to persist a state snapshot
2024-06-13T18:06:03.715Z [ERROR] vertex "module.api_gateway.aws_api_gateway_method_settings.endpoint_throttle[\"path_get\"]" error: reading API Gateway Method Settings (<gatewayid>-<env>-path{proxy+}/GET): empty result
2024-06-13T18:06:03.715Z [TRACE] vertex "module.api_gateway.aws_api_gateway_method_settings.endpoint_throttle[\"path_get\"]": visit complete, with errors
2024-06-13T18:06:03.715Z [TRACE] dag/walk: upstream of "module.api_gateway (close)" errored, so skipping']

�[31m╷�[0m�[0m
�[31m│�[0m �[0m�[1m�[31mError: �[0m�[0m�[1mreading API Gateway Method Settings (<gatewayid>-<env>-path/{proxy+}/GET): empty result�[0m
�[31m│�[0m �[0m
�[31m│�[0m �[0m�[0m  with module.api_gateway.aws_api_gateway_method_settings.endpoint_throttle["path_get"],
�[31m│�[0m �[0m  on ../../modules/aws/api_gateway/main.tf line 76, in resource "aws_api_gateway_method_settings" "endpoint_throttle":
�[31m│�[0m �[0m  76: resource "aws_api_gateway_method_settings" "endpoint_throttle" �[4m{�[0m�[0m
�[31m│�[0m �[0m
�[31m╵�[0m�[0m
2024-06-13T18:06:06.115Z [DEBUG] provider.stdio: received EOF, stopping recv loop: err="rpc error: code = Unavailable desc = error reading from server: EOF"
2024-06-13T18:06:06.130Z [DEBUG] provider: plugin process exited: path=.terraform/providers/registry.terraform.io/hashicorp/aws/5.53.0/linux_amd64/terraform-provider-aws_v5.53.0_x5 pid=29256
2024-06-13T18:06:06.130Z [DEBUG] provider: plugin exited
Operation failed: failed running terraform apply (exit 1)�

Panic Output

No response

Important Factoids

This was working with the following configurations:
Terraform 1.6.3 / AWS Provider 5.39.1
Terraform 1.8.5 / AWS Provider 5.39.1

This failed with:
Terraform 1.8.5 / AWS Provider 5.46.0
Terraform 1.8.5 / AWS Provider 5.53.0

Although this does work, in earlier versions of terraform, the resource is almost always shown as "tainted" and is always replaced.

References

No response

Would you like to implement a fix?

None

[github-actions]

Community Note

Voting for Prioritization

• Please vote on this issue by adding a 👍 reaction to the original post to help the community and maintainers prioritize this request.
• Please see our prioritization guide for information on how we prioritize.
• Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request.

Volunteering to Work on This Issue

• If you are interested in working on this issue, please leave a comment.
• If this would be your first contribution, please review the contribution guide.

[justinretzolk]

Possibly related #13612

[mborchuk]

It does not work with versions Terraform 1.8.4 / AWS Provider 5.54.0 as well

[EEsteban2001]

#37959

[AndrewReaganM]

I ran into this issue and was surprised to see that adding the stage name to the beginning of the method path fixes it. I wouldn't have thought that would be necessary.

So for the example given in the original report, the method path would be:

method_path = "develop/path/{proxy+}/GET"

In this case, develop is the name of the stage.

EDIT: This appears to work, but doesn't actually change the method settings for your API. There seems to be something broken in both the AWS service and possibly in the AWS Terraform provider as sometimes I can't even get the AWS Console to handle this properly.

[Roman-Banakh]

I just checked with different versions of the AWS provider and found that the issue started reproducing in version 5.45.0. However, if the version is lower than 5.45.0, the resource aws_api_gateway_method_settings passes the creation. However, if you run it again, Terraform will try to create it again.

[Roman-Banakh]

In my case, the issue was fixed after aws_api_gateway_method resource redeploy

[InvokedLambda]

Im facing the same issue, since 5.45.0 i get "empty result" as an exception.

Adding the stage name to the path for versions >=5.45.0 what @AndrewReaganM said does not work for me.

[AnitaErnszt]

We have downgraded to 5.41 at it fixed the issue for us.
Once deployed, you can upgrade back to the current version, and it should continue to work.

[AnitaErnszt]

For ref, I believe this was caused by the AWS SDK upgrade from v1 -> v2.

Based on CloudTrail logs it seems v2 replaces / prefix with ~1 in the output (v1 just stripped the / prefix), which return in terraform unable to find the key it's looking for in the response