Destroying and rebuilding firewall, despite no changes

[sjackson0109]

Is there an existing issue for this?

• I have searched the existing issues

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

Terraform Version

1.4.5 x64

AzureRM Provider Version

3.41.0

Affected Resource(s)/Data Source(s)

azurerm_firewall

Terraform Configuration Files

fw_name     = "UKSFW"
fw_sku_tier = "Standard"
fw_mgmt_subnet_prefix = ["10.1.1.0/25"]
fw_mgmt_subnet_name   = "AzureFirewallManagementSubnet"
fw_subnet_prefix      = ["10.1.1.128/25"]
fw_subnet_name        = "AzureFirewallSubnet"

fw_public_int_01 = {
  name              = "UKSFW-PIP-01"
  allocation_method = "Static"
  sku               = "Standard"
}
fw_public_int_02 = {
  name              = "UKSFW-PIP-02"
  allocation_method = "Static"
  sku               = "Standard"
}
fw_public_int_MGMT = {
  name              = "UKSFW-PIP-MGMT"
  allocation_method = "Static"
  sku               = "Standard"
}
product_public_ip = {
  PRODUCT_DEV = {
    fqdn              = "dev.company.com"
    sku               = "Standard"
    allocation_method = "Static"
    ip_version        = "IPv4"
    zones             = ["1", "2", "3"]
  }
  PRODUCT_TEST = {
    fqdn              = "test.company.com"
    sku               = "Standard"
    allocation_method = "Static"
    ip_version        = "IPv4"
    zones             = ["1", "2", "3"]
  }
}
# Management Public IP
resource "azurerm_public_ip" "fw_public_int_MGMT" {
  depends_on          = [azurerm_resource_group.uks_hub_rg]
  name                = var.fw_public_int_MGMT.name
  location            = azurerm_resource_group.uks_hub_rg.location
  resource_group_name = azurerm_resource_group.uks_hub_rg.name
  allocation_method   = var.fw_public_int_MGMT.allocation_method
  sku                 = var.fw_public_int_MGMT.sku
}
# Device Public IP (in/outbound routing)
resource "azurerm_public_ip" "fw_public_int_01" {
  depends_on          = [azurerm_resource_group.uks_hub_rg]
  name                = var.fw_public_int_01.name
  location            = azurerm_resource_group.uks_hub_rg.location
  resource_group_name = azurerm_resource_group.uks_hub_rg.name
  allocation_method   = var.fw_public_int_01.allocation_method
  sku                 = var.fw_public_int_01.sku
}
resource "azurerm_public_ip" "fw_public_int_02" {
  depends_on          = [azurerm_resource_group.uks_hub_rg]
  name                = var.fw_public_int_02.name
  location            = azurerm_resource_group.uks_hub_rg.location
  resource_group_name = azurerm_resource_group.uks_hub_rg.name
  allocation_method   = var.fw_public_int_02.allocation_method
  sku                 = var.fw_public_int_02.sku
}
# Application Public IP addresses
resource "azurerm_public_ip" "uks_product" {
  depends_on          = [azurerm_resource_group.uks_hub_rg]
  for_each            = var.product_public_ip
  name                = each.value.fqdn
  location            = azurerm_resource_group.uks_hub_rg.location
  resource_group_name = azurerm_resource_group.uks_hub_rg.name
  sku                 = try(each.value.sku, "Standard")
  allocation_method   = try(each.value.allocation_method, "Static")
  ip_version          = try(each.value.ip_version, "IPv4")
  zones               = try(each.value.zones, ["1"])
  domain_name_label   = replace(lower("${each.value.fqdn}"), ".", "-")
  lifecycle {
    create_before_destroy = true
  }
}
# FIREWALL OBJECT
resource "azurerm_firewall" "uks" {
  depends_on = [
    azurerm_resource_group.uks_hub_rg,
    azurerm_public_ip.fw_public_int_01,
    azurerm_public_ip.fw_public_int_02,
    azurerm_public_ip.uks_product,
    azurerm_firewall_policy.uks_policy
  ]
  #    
  name                = var.fw_name
  location            = azurerm_resource_group.uks_hub_rg.location
  resource_group_name = azurerm_resource_group.uks_hub_rg.name
  sku_name            = "AZFW_VNet"
  sku_tier            = var.fw_sku_tier
  #Mgmt interface comes first
  management_ip_configuration {
    name                 = azurerm_public_ip.fw_public_int_MGMT.name
    subnet_id            = azurerm_subnet.subnets["AzureFirewallManagementSubnet"].id
    public_ip_address_id = azurerm_public_ip.fw_public_int_MGMT.id
  }
  #Public Interface(s) must be attached next (HA pair = 2x Public IPs required)
  ip_configuration {
    name                 = azurerm_public_ip.fw_public_int_01.name
    subnet_id            = azurerm_subnet.subnets["AzureFirewallSubnet"].id
    public_ip_address_id = azurerm_public_ip.fw_public_int_01.id
  }
  ip_configuration {
    name = azurerm_public_ip.fw_public_int_02.name
    #subnet_id                                     = azurerm_subnet.subnets["AzureFirewallSubnet"].id
    public_ip_address_id = azurerm_public_ip.fw_public_int_02.id
  }
  # Attach the Public IPs for the Product to use
  dynamic "ip_configuration" {
    for_each = azurerm_public_ip.uks_product
    content {
      name                 = ip_configuration.value.name
      public_ip_address_id = ip_configuration.value.id
    }
  }

  firewall_policy_id = azurerm_firewall_policy.uks_policy.id
}

Debug Output/Panic Output

No panic

   # azurerm_firewall.uks must be replaced
  -/+ resource "azurerm_firewall" "uks" {
      - dns_servers         = [] -> null
      ~ id                  = "/subscriptions/<GUID REMOVED>/resourceGroups/UKSRG01/providers/Microsoft.Network/azureFirewalls/UKSFW" -> (known after apply)
        name                = "UKSFW"
      - private_ip_ranges   = [] -> null
      - tags                = {} -> null
      ~ threat_intel_mode   = "Alert" -> (known after apply)
      - zones               = [] -> null
        # (5 unchanged attributes hidden)

      ~ ip_configuration {
            name                 = "UKSFW-PIP-01"
          ~ private_ip_address   = "10.1.1.4" -> (known after apply)
            # (2 unchanged attributes hidden)
        }
      ~ ip_configuration {
            name                 = "UKSFW-PIP-02"
          + private_ip_address   = (known after apply)
            # (1 unchanged attribute hidden)
        }
      ~ ip_configuration {
            name                 = "dev.company.com"
          + private_ip_address   = (known after apply)
            # (1 unchanged attribute hidden)
        }
      ~ ip_configuration {
            name                 = "test.company.com"
          + private_ip_address   = (known after apply)
            # (1 unchanged attribute hidden)
        }

Expected Behaviour

No replacement of the appliance. This takes the network offline for upto 12 minutes whilst it's deleted and rebuilt!

Actual Behaviour

azurerm_firewall.uks must be replaced

-/+ resource "azurerm_firewall" "uks" {
- dns_servers = [] -> null
~ id = "/subscriptions//resourceGroups/UKSRG01/providers/Microsoft.Network/azureFirewalls/UKSFW" -> (known after apply)
name = "UKSFW"
- private_ip_ranges = [] -> null
- tags = {} -> null
~ threat_intel_mode = "Alert" -> (known after apply)
- zones = [] -> null
# (5 unchanged attributes hidden)

  ~ ip_configuration {
        name                 = "UKSFW-PIP-01"
      ~ private_ip_address   = "10.1.1.4" -> (known after apply)
        # (2 unchanged attributes hidden)
    }
  ~ ip_configuration {
        name                 = "UKSFW-PIP-02"
      + private_ip_address   = (known after apply)
        # (1 unchanged attribute hidden)
    }
  ~ ip_configuration {
        name                 = "dev.company.com"
      + private_ip_address   = (known after apply)
        # (1 unchanged attribute hidden)
    }
  ~ ip_configuration {
        name                 = "test.company.com"
      + private_ip_address   = (known after apply)
        # (1 unchanged attribute hidden)
    }

Steps to Reproduce

terraform plan
terraform apply

Important Factoids

None

References

No associations i have found that closely relate to this issue.

[aristosvo]

Hi @sjackson0109! Can you provide the plan output of the whole resource?

Because I cannot see the whole plan it is not possible to locate the property that triggers the replacement.

[sjackson0109]

This is a customer pilot environment, so of course contains private information. I have to be careful what data I share online. I can say the ‘terraform plan’ section clearly outputs the firewall is being replaced, I would have expected the following statement indicating what causes the replacement `# forces replacement`. Am guessing that is what you are looking for? I will however perform `terraform plan -out $file`; and have a second pass at reviewing the output asap

[sjackson0109]

I think i got to the bottom of it - the Dynamic Block of interface IPs; for_each of the var.product_public_ips was the cause.
I hard-coded the for_each line to work only with a single public IP for_each = azurerm_public_ip.uks_product[0], terraform apply did NOT replace the firewall that time. soo odd.

Interesting these azurerm_public_ip objects all have resource locks on them in azure; so the public IPs are not getting replaced.

# Application Public IP addresses
resource "azurerm_public_ip" "uks_product" {
  depends_on          = [azurerm_resource_group.uks_hub_rg]
  for_each            = var.product_public_ip
  name                = each.value.fqdn
  location            = azurerm_resource_group.uks_hub_rg.location
  resource_group_name = azurerm_resource_group.uks_hub_rg.name
  sku                 = try(each.value.sku, "Standard")
  allocation_method   = try(each.value.allocation_method, "Static")
  ip_version          = try(each.value.ip_version, "IPv4")
  zones               = try(each.value.zones, ["1"])
  domain_name_label   = replace(lower("${each.value.fqdn}"), ".", "-")
  lifecycle {
    create_before_destroy = true
  }
}

resource "azurerm_firewall" "uks" {
 #####
   # Attach the Public IPs for the Product to use
  dynamic "ip_configuration" {
    for_each = azurerm_public_ip.uks_product
    content {
      name                 = ip_configuration.value.name
      public_ip_address_id = ip_configuration.value.id
    }
  }
 }

I've decided to abandon the Application Gateway AFTER Firewall in favour of the Application Gateway BEFORE Firewall - for the simple fact, the WebApp looses the Public IP address of customers visiting the websites.