emr: terraform state containing very old terminated cluster causes plan to fail

This issue was originally opened by @steliospaps as hashicorp/terraform#20531. It was migrated here as a result of the provider split. The original body of the issue is below.

---

Terraform v0.11.7

• Downloading plugin for provider "aws" (2.0.0)

I cannot easily test with 0.11.11 because I do not own the build server where this runs.

EMR cluster was manually terminated in dec 10th 2018. When I run terraform plan in feb 28th 2019 to create it, it failed.

I believe it failed because the EMR cluster history in aws goes back only two months, so the lookup for the cluster fails.

Workaround: I had no other resources in that terraform config, so I deleted the statefile. If I had other resources I would not be able to do that.

output (anonymised):

Refreshing Terraform state in-memory prior to plan...
The refreshed state will be used to calculate this plan, but will not be
persisted to local or remote state storage.

....

Error: Error refreshing state: 1 error(s) occurred:

• aws_emr_cluster.XXXXXXXXX: 1 error(s) occurred:

• aws_emr_cluster.XXXXXXXX: aws_emr_cluster.XXXXXXXX: Error reading EMR cluster: InvalidRequestException: Cluster id 'j-XXXXXXXXXX' is not valid.
  status code: 400, request id: XXXXXXX-XXXXXX-XXXX-XXXX-XXXXXXXXX

[bflad]

Hi @steliospaps 👋 Thanks for reporting this and sorry for the trouble. This is certainly something that can be addressed by removing the resource during refresh, e.g.

if isAWSErr(err, emr.ErrCodeInvalidRequestException, "is not valid") {
  log.Printf("[DEBUG] EMR Cluster (%s) not found, removing from state", d.Id())
  d.SetId("")
  return nil
}

We have some currently in-flight bugfix work occurring right now with that resource (#7405) so it may get included with that or can be fixed right afterwards.

[ImDevinC]

Was a solution for this ever implemented? You can fix this without fully destroying the state by using terraform state rm aws_emr_cluster.XXXXXX, but not having to do that would be great

[klsnreddy]

Is there a fix for this, apart from removing that resource from state file, I am getting this exception while I am trying to destroy terrafrom resource also, my cluster is Active and I need to terminate the cluster, when I say 'terraform destroy', I am getting this same exception.

[chernetsov]

Bumping this. We've stumbled into this too. Doing terraform state rm aws_emr_cluster ... for now.

[bflad]

A best effort fix for this has been merged and will release with version 3.23.0 of the Terraform AWS Provider, likely tomorrow. Thanks to @apetresc for the implementation and testing. 👍

This has been released in version 3.23.0 of the Terraform AWS provider. Please see the Terraform documentation on provider versioning or reach out if you need any assistance upgrading.

For further feature requests or bug reports with this functionality, please create a new GitHub issue following the template for triage. Thanks!

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you feel this issue should be reopened, we encourage creating a new issue linking back to this one for added context. Thanks!