terraform plan does not account for changes being applied to existing resources which results in a failure

[FreyMo]

Terraform Version

Terraform v0.15.0
on windows_amd64
+ provider registry.terraform.io/hashicorp/azuread v1.4.0
+ provider registry.terraform.io/hashicorp/azurerm v2.55.0

Terraform Configuration Files

terraform {
  required_providers {
    azurerm = {
      source  = "hashicorp/azurerm"
      version = "=2.55.0"
    }
    azuread = {
      source  = "hashicorp/azuread"
      version = "=1.4.0"
    }
  }
}

provider "azurerm" {
  features {}
}

locals {
  prefix = "reproducecore"
}

data "azurerm_client_config" "current" {}

resource "azurerm_resource_group" "this" {
  name     = "rg-${local.prefix}"
  location = "westeurope"
}

resource "azurerm_storage_account" "this" {
  name                     = "sa${local.prefix}"
  resource_group_name      = azurerm_resource_group.this.name
  location                 = azurerm_resource_group.this.location
  account_tier             = "Standard"
  account_replication_type = "LRS"

  # identity {
  #   type = "SystemAssigned"
  # }
}

resource "azurerm_key_vault" "this" {
  name                = "kv${local.prefix}"
  resource_group_name = azurerm_resource_group.this.name
  location            = azurerm_resource_group.this.location
  tenant_id           = data.azurerm_client_config.current.tenant_id
  sku_name            = "standard"
}

# resource "azurerm_key_vault_access_policy" "this" {
#   key_vault_id = azurerm_key_vault.this.id
#   tenant_id    = data.azurerm_client_config.current.tenant_id
#   object_id    = azurerm_storage_account.this.identity[0].principal_id

#   secret_permissions = ["Get"]
# }

Error Output

azurerm_resource_group.this: Refreshing state... [id=/subscriptions/<uuid>/resourceGroups/rg-reproducecore]
azurerm_key_vault.this: Refreshing state... [id=/subscriptions/<uuid>/resourceGroups/rg-reproducecore/providers/Microsoft.KeyVault/vaults/kvreproducecore]
azurerm_storage_account.this: Refreshing state... [id=/subscriptions/<uuid>/resourceGroups/rg-reproducecore/providers/Microsoft.Storage/storageAccounts/sareproducecore]
╷
│ Error: "object_id": required field is not set
│
│   on main.tf line 49, in resource "azurerm_key_vault_access_policy" "this":
│   49: resource "azurerm_key_vault_access_policy" "this" {
│
╵

Expected Behavior

Adding SystemAssigned Managed Identity to an existing resource should update its exported identity attribute to contain one entry. Therefore it should be possible to apply the provided configuration without any problems. azurerm_key_vault_access_policy relies on the Managed Identity exported from the azurerm_storage_account.

Actual Behavior

Applying was not possible because accessing the exported identity attribute failed. As a result the object_id of the azurerm_key_vault_acces_policy can not be set. Output as provided in Error Output is shown.

Steps to Reproduce

1. terraform init && terraform apply
2. uncomment the commented-out code
3. terraform plan

Additional Context

In the References there are two issues linked to the azurerm provider that brought me here. It is claimed that this is not a provider problem but rather a problem of Terraform core. It affects multiple resources in the azurerm provider and is not confined to azurerm_storage_account. Any addition of a SystemAssigned Managed Identity to an existing resource will prevent us from using that identity in the same configuration.

Possible workarounds

• Create the entire configuration from scratch. Then everything works
• Do this in a multi-step process
  • First of all add the Managed Identity to an existing resource and apply
  • Add the azurerm_key_vault_access_policy that can now safely access identity[0]

References

• Error when adding azurerm_app_service.identity and azurerm_role_assignment to existing infrastructure terraform-provider-azurerm#4532
• Identity is empty list of objects for access_policy terraform-provider-azurerm#11022

[jbardin]

Hi @FreyMo,

Thanks for filing the issue here. It is up to the provider to mark any planned values which will not be known until apply as "unknown". In this case the provider is planning a null value for principal_id, which terraform is then using within azurerm_key_vault_access_policy.

The "provider" from Terraform's perspective also includes the behavior of the SDK used by the provider. In this case the behavior seen is a shortcoming of the legacy provider SDK which did not have the ability to handle many situations with complex types, and nested attributes.

Since this is a known issue with the legacy SDK, and there is work being done on a next generation SDK already, I'm going to transfer this to the azure provider. They may choose to not work around the issue until #28340 is in place, but the solution does need to come from the provider or SDK in some way.

Thanks!

[FreyMo]

Hey @jbardin,

thank you for the quick reply and the detailed clarification. I really appreciate it.

Right now we are kind of stuck between a rock and a hard place because as the end customer, we cannot work around this. Could someone of the azurerm provider team comment on this? Maybe @tombuildsstuff?

Thanks in advance

[ghost]

This issue has been automatically migrated to hashicorp/terraform-provider-azurerm#11374 because it looks like an issue with that provider. If you believe this is not an issue with the provider, please reply to hashicorp/terraform-provider-azurerm#11374.

[magodo]

@jbardin Do you have any idea how does the provider workaround this issue? Seems using ResourceDiff.SetNewComputed() in the CustomizeDiff doesn't help here.

[jbardin]

@magodo, yes, it appears that is also a known issue with the SDK: hashicorp/terraform-plugin-sdk#459. I don't think there is a workaround other than using multiple apply steps to converge.

[github-actions]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.
If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.