-
Notifications
You must be signed in to change notification settings - Fork 4.6k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Creating new firewall rule with azurerm_firewall_policy_rule_collection_group force existing rule to be recreated. #18114
Comments
Hi @henryxuteck just a simple confirmation: is azurerm 2.x works as your expect and from 3.x azurerm raised the recreate action? |
Yes, 2.0 works fine, 3.0 doesn't. |
Hi @juicybaba, I have tried your configuration by adding rule3 after rule2 and don't see the recreation of rule2. or how can we tell it is In the update of azurerm 3.0, the Firewall changes from Set to List should not cause this kind of behavior |
@wuxu92 thanks for getting back to me, sorry for my bad example, I think the root cause is that no matter which order network rule get passed in, the network rule will be re-ordered alphabetically. e.g. when appending And again, this happens in My interpretation of Firewall: The behavior of the nested items will be changed to List instead of Sets where required, meaning that |
here is the origin PR of the change from |
@wuxu92 thanks for getting back to me. Please see the full configuration for Perhaps the interation in the Before:
After:
|
Has anyone found a solution for this issue? |
We have the same issue here. As we can see in Azure portal, each time when we create Rule collection it has its own specific unpredictable rules and collections order which cannot be changed and if this order do not match the terraform defined one - it tries to change it. We have terraform module for
Then, if you delete manually collection group from the portal and run terraform again it can be different again.
But based on our observation of Azure Firewall Policy service each new rule/collection/collection_group is always appended (added at the end) not matter what the name/type is. Thus we think this is option 1 issue |
There are several issues open with Hashicorp for the azurerm_firewall_policy_rule_collection_group module. As I've worked on this issue and talked to Microsoft reps about it I think the issue is with how Microsoft designed Firewall Policy Rule Collection Groups and not an issue with how Terraform deploys them. https://github.com/hashicorp/terraform-provider-azurerm/issues/16586 Microsoft made the decision with the Rule Collection Groups to make the list of collections and subsequent rules an ordered list rather than making each collection a resource. This means that there isn't a way for Azure to identify each collection and thus if the list changes it must recreate the list. Also notice in Azure Portal that the Rule Collections in a Rule Collection Group are ordered in reverse order of which they are processed (Application, then Network, then NAT). To reduce the potential impact to our deployments we broke the collections into individual yml files, and when we pull the collections together we sort them by priority. There will still be periodic changes to the list as we clean up old rules, but it is a higher probability that new rules will be placed at the bottom of the priority list. Microsoft has also confirmed with me that when updates are made to the firewall rules, the instance of the firewall will be removed from the pool, have the updates made to it, and then the instance will be returned to the pool before moving to the next firewall instance (a 1-2 minute process per instance). So any restructuring of the collections and rules will not impact production. |
Hi @jr8279 just checking if I got this correctly, There should be no downtime on the network connectivity even if the collections groups and rules are being recreated? |
According to Microsoft, there shouldn’t be a downtime. But depending on how many firewalls instances you are using, there could be a period of time where some traffic is using the new rule list and some traffic is using the old rule list. They told me it takes 1-2 minutes per instance to update. |
@juicybaba The cause should be the
the output in Terraform console, the key will be reordered:
So If we want the rule_collection to keep the order, we have to define the local variable as a list and then loop over the list. a local variable like below(changing
and the new resource configuration to consume a list in dynamic collection blocks, Then append new items to the list will not display recreating operation in terraform plan (but this plane output does not indicate there is no replace/recreate in the Azure firewall server side as comments above give more details): resource "azurerm_firewall_policy_rule_collection_group" "default2" {
for_each = local.firewall_rule_collection_groups_list
name = format("fwrcg-%s", each.key)
firewall_policy_id = azurerm_firewall_policy.firewall_test.id
priority = each.value.priority
dynamic "network_rule_collection" {
iterator = self
for_each = {for idx, v in each.value.network_rule_collections: idx => v}
content {
name = format("nrc-%s", self.value.key)
priority = self.value.priority
action = self.value.action
dynamic "rule" {
for_each = self.value.rules
content {
name = rule.key
protocols = rule.value.protocols
destination_ports = rule.value.destination_ports
destination_addresses = rule.value.destination_addresses
destination_ip_groups = rule.value.destination_ip_groups
destination_fqdns = rule.value.destination_fqdns
source_addresses = rule.value.source_addresses
source_ip_groups = rule.value.source_ip_groups
}
}
}
}
} |
Is there an existing issue for this?
Community Note
Terraform Version
1.1.9
AzureRM Provider Version
3.19.1
Affected Resource(s)/Data Source(s)
azurerm_firewall_policy_rule_collection_group
Terraform Configuration Files
Debug Output/Panic Output
Expected Behaviour
rule 3 should be added without impacting existing rules.
This can be the cause, but I attempted to find a way to insert the rule3 to the end of the list by what I mentioned earlier, but I couldnt make it work.
Actual Behaviour
No response
Steps to Reproduce
No response
Important Factoids
No response
References
No response
The text was updated successfully, but these errors were encountered: