[Bug]: aws_s3_bucket data sources cannot be destroyed if the bucket no longer exists

[josephmidura]

Terraform Core Version

1.5.3

AWS Provider Version

4.67.0

Affected Resource(s)

data.aws_s3_bucket

Expected Behavior

A warning is displayed and terraform destroy continues to remove the resource as normal.

Actual Behavior

The aws_s3_bucket data source is unable to delete during terraform destroy if the associated bucket is already deleted.

Relevant Error/Panic Output Snippet

No response

Terraform Configuration Files

data "aws_s3_bucket" "glue" {
  bucket = "${local.project_stage}-glue" 
}

Steps to Reproduce

The terraform destroy was run on a project. The destroy operations stopped with an error because deletion protection was enabled on an RDS database. After deletion protection was disabled in the console for the RDS database, terraform destroy command was run again.

The aws_s3_bucket data source now references a bucket that no longer exists (was deleted during the initial terraform destroy) and the destroy operation failed with the following error:

Error: Failed getting S3 bucket (bucket-name): NotFound: Not Found

terraform state list does not show the bucket resource.

Debug Output

No response

Panic Output

No response

Important Factoids

terraform installed via asdf

References

No response

Would you like to implement a fix?

None

[github-actions]

Community Note

Voting for Prioritization

• Please vote on this issue by adding a 👍 reaction to the original post to help the community and maintainers prioritize this request.
• Please see our prioritization guide for information on how we prioritize.
• Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request.

Volunteering to Work on This Issue

• If you are interested in working on this issue, please leave a comment.
• If this would be your first contribution, please review the contribution guide.

[justinretzolk]

Hey @josephmidura 👋 Thank you for taking the time to raise this! Can you give me an idea of what local.project_stage looks like? I'm also curious -- are you creating a bucket using an aws_s3_bucket resource and then reading it using the corresponding data source in the same configuration?

Unfortunately, I don't believe there's a way to skip a data source being read during a destroy (Terraform assumes you need the most up to date information possible), so this might be a bit tricky to get around.

[josephmidura]

Thanks for the reply @justinretzolk. Here is some additional code that helps explain local.project_stage:

variable "project" {
  description = "Project name"
  default     = "name"
}

variable "stage" {
  type        = string
  description = "Stage"
  default     = "prod"
}

locals {
  project_stage = "${var.project}-${var.stage}"
  region        = data.aws_region.current.name
  account_id    = data.aws_caller_identity.current.account_id
}

So the code snippet I included above could be rewritten as the following:

data "aws_s3_bucket" "glue" {
  bucket = "${var.project}-${var.stage}-glue" 
}

or

data "aws_s3_bucket" "glue" {
  bucket = "name-prod-glue" 
}

Yes, I created the name-prod-glue bucket using an aws_s3_bucket resource and the aws_s3_bucket data in the same configuration.

Workaround that was successful in my case

Today, I created the name-prod-glue bucket manually, ran terraform destroy again, and this time all resources deleted, including data.aws_s3_bucket. I'm happy that all resources were cleanly destroyed, but this workaround is not a robust solution. I'll appreciate any insight you can offer.

[justinretzolk]

Hey @josephmidura 👋 Thank you for the additional insight! The only thing that still stands out to me is the use of the data source paired with the resource as well. I'm fairly confident that doing so is what caused this issue, and I'm having a hard time understanding why you would need to do so. Are you able to give me a better idea of why that's needed?

As an aside, I'd not tested using terraform state rm on a data source before, but apparently it works! If you find yourself in this situation again, that's probably an easier solution than needing to create the bucket again, etc.

$ terraform state list
data.null_data_source.this

$ terraform state rm data.null_data_source.this
Removed data.null_data_source.this
Successfully removed 1 resource instance(s).

$ terraform state list

[josephmidura]

Hi @justinretzolk thanks for the suggestion. When I ran terraform state list previously there was no data souce that showed up in the list, so I wasn't able to use the terraform state rm option. I am using terraform 1.5.3 and it's possible I need a later version for the functionality you mentioned.

In my use case, there is an s3 bucket used with a data lake. The goal is to register a lake formation resource (S3) to the data lake. Here are the relevant parts of the code:

variable "project" {
  description = "Project name"
  default     = "name"
}

variable "stage" {
  type        = string
  description = "Stage"
  default     = "prod"
}

locals {
  project_stage = "${var.project}-${var.stage}"
  region        = data.aws_region.current.name
  account_id    = data.aws_caller_identity.current.account_id
}

resource "aws_s3_bucket" "glue" {
  bucket = "${local.project_stage}-glue" 
}

data "aws_s3_bucket" "glue" {
  bucket = "${local.project_stage}-glue" 
}

resource "aws_lakeformation_resource" "glue-bucket-as-location" {
  arn = data.aws_s3_bucket.glue.arn
}

[justinretzolk]

Thanks for the additional information here @josephmidura! Based on your sample configuration, this should be a fairly easy fix.

Right now, there's no dependency between aws_s3_bucket.glue and data.aws_s3_bucket.glue, so there's no guarantee that creation/deletion of those two will happen in an order that will actually work. The data source could be attempted to be read before it's actually created, or after it's been deleted (as happened in your initial report).

On the other hand, the aws_s3_bucket resource has an arn attribute, so it's not necessary to use the related data source to do a separate lookup in order to retrieve that information. Refactoring your example configuration would look like:

variable "project" {
  description = "Project name"
  default     = "name"
}

variable "stage" {
  type        = string
  description = "Stage"
  default     = "prod"
}

locals {
  project_stage = "${var.project}-${var.stage}"
  region        = data.aws_region.current.name
  account_id    = data.aws_caller_identity.current.account_id
}

resource "aws_s3_bucket" "glue" {
  bucket = "${local.project_stage}-glue" 
}

resource "aws_lakeformation_resource" "glue-bucket-as-location" {
  arn = aws_s3_bucket.glue.arn
}