Merge pull request hashicorp#259 from hashicorp/add-azure-functions

update enforce-mandatory-tags policies

Author: rberlind

governance/third-generation/README.md

@@ -33,7 +33,7 @@ You can find most of the common functions used in the third-generation policies
   * [tfconfig-functions](./common-functions/tfconfig-functions)
   * [tfrun-functions](./common-functions/tfrun-functions)
 
-There are also some functions used to validate assumed roles for the AWS provider in [aws-functions](./aws/aws-functions).
+There are also some functions that can be used with the AWS and Azure providers in [aws-functions](./aws/aws-functions) and [azure-functions](./azure/azure-functions).
 
 Unlike the second-generation common functions that were each defined in a separate file, all of the common functions that use any of the 4 Terraform Sentinel imports (tfplan/v2, tfstate/v2, tfconfig/v2, and tfrun) are defined in a single file. This makes it easier to import all of the functions that use one of those imports into the Sentinel CLI test cases and Terraform Cloud policy sets, since those only need a single stanza such as this one for each module:
 ```
@@ -43,7 +43,7 @@ Unlike the second-generation common functions that were each defined in a separa
   }
 }
 ```
-Test cases that use the other modules would either change all three occurrences of "tfplan" in that stanza to "tfstate", "tfconfig", or "tfrun" or would add additional stanzas with those changes.
+Test cases that use the other modules would either change all three occurrences of "tfplan" in that stanza to "tfstate", "tfconfig", "tfrun", "aws", or "azure" or would add additional stanzas with those changes.
 
 We have put each Sentinel module in its own directory which also contains Markdown files for each of the module's functions under a docs directory. Each of these Markdown files describes the function, its declaration, its arguments, other common functions it uses, what it returns, and what it prints. It also gives examples of calling the function and sometimes lists some policies that call it.
 
@@ -56,8 +56,9 @@ import "tfstate-functions" as state
 import "tfconfig-functions" as config
 import "tfrun-functions" as run
 import "aws-functions" as aws
+import "azure-functions" as azure
 ```
-In this case, we are using `plan`, `state`, `config`, `run`, and `aws` as aliases for the five imports to keep lines that use their functions shorter. Of course, you only need to import the modules that contain functions that your policy actually calls.
+In this case, we are using `plan`, `state`, `config`, `run`, `aws`, and `azure` as aliases for the six imports to keep lines that use their functions shorter. Of course, you only need to import the modules that contain functions that your policy actually calls.
 
 ### The Functions of the tfplan-functions and tfstate-functions Modules
 We discuss these two modules together because they are essentially identical except for their use of the tfplan/v2 and tfstate/v2 imports.
@@ -106,7 +107,7 @@ Documentation for each individual function can be found in this directory:
 
 ### The Functions of the aws-functions Module
 The `aws-functions` module (which is located under in the aws/aws-functions directory) has the following functions:
-  * The `find_resources_with_standard_tags` function finds all AWS resources that use standard AWS tags in the current plan that are being created or modified.
+  * The `find_resources_with_standard_tags` function finds all AWS resources of specified types that should have tags in the current plan that are not being permanently deleted.
   * The `determine_role_arn` function determines the ARN of a role set in the `role_arn` parameter of an AWS provider. It can only determine the role_arn if it is set to either a hard-coded value or to a reference to a single Terraform variable. It sets the role to "complex" if it finds a single non-variable reference or if it finds multiple references. It sets the role to "none" if no role arn is found.
   * The `get_assumed_roles` function gets all roles assumed by AWS providers in the current Terraform configuration. It calls the `determine_role_arn` function.
   * The `validate_assumed_roles_with_list` function validates assumed roles found by the `get_assumed_roles` function against a list of role ARNs.
@@ -115,6 +116,13 @@ The `aws-functions` module (which is located under in the aws/aws-functions dire
 Documentation for each individual function can be found in this directory:
   * [aws-functions](./aws/aws-functions/docs)
 
+### The Functions of the azure-functions Module
+The `azure-functions` module (which is located under in the azure/azure-functions directory) has the following functions:
+  * The `find_resources_with_standard_tags` function finds all Azure resources of specified types that should have tags in the current plan that are not being permanently deleted.
+
+Documentation for each individual function can be found in this directory:
+  * [azure-functions](./azure/azure-functions/docs)
+
 ## Mock Files and Test Cases
 Sentinel [mock files](https://www.terraform.io/docs/enterprise/sentinel/mock.html) and [test cases](https://docs.hashicorp.com/sentinel/commands/config#test-cases) have been provided under the test directory of each cloud so that all the policies can be tested with the [Sentinel CLI](https://docs.hashicorp.com/sentinel/commands). The mocks were generated from actual Terraform 0.12 plans run against Terraform code that provisioned resources in these clouds. The pass and fail mock files were edited to respectively pass and fail the associated Sentinel policies. Some policies, including those that have multiple rules, have multiple fail mock files with names that indicate which condition or conditions they fail.
 

governance/third-generation/aws/aws-functions/aws-functions.sentinel

@@ -10,12 +10,13 @@ import "types"
 ##### Functions #####
 
 ### find_resources_with_standard_tags ###
-find_resources_with_standard_tags = func() {
+find_resources_with_standard_tags = func(resource_types) {
   resources = filter tfplan.resource_changes as address, rc {
     rc.provider_name matches "(.*)aws$" and
-    rc.type not in ["aws_autoscaling_group"] and
+    rc.type in resource_types and
   	rc.mode is "managed" and
-  	(rc.change.actions contains "create" or rc.change.actions contains "update")
+    (rc.change.actions contains "create" or rc.change.actions contains "update" or
+     rc.change.actions contains "read" or rc.change.actions contains "no-op")
   }
 
   return resources

governance/third-generation/aws/aws-functions/docs/find_resources_with_standard_tags.md

@@ -1,18 +1,21 @@
 # find_resources_with_standard_tags
-This function finds all resource instances for the AWS provider that use standard AWS tags in the current plan that are being created or modified using the [tfplan/v2](https://www.terraform.io/docs/cloud/sentinel/import/tfplan-v2.html) import.
+This function finds all AWS resource instances of specified types in the current plan that are not being permanently deleted using the [tfplan/v2](https://www.terraform.io/docs/cloud/sentinel/import/tfplan-v2.html) import.
 
-Currently, the only AWS resource excluded is `aws_autoscaling_group`. If you discover other AWS resources that do not use the `tags` attribute in the standard way, then add them to the list that already includes `aws_autoscaling_group`.
+It was updated on 9/29/2020 to work with both the short name of the AWS provider, "aws", and fully-qualfied provider names that match the regex, `(.*)aws$`. This was required because Terraform 0.13 and above returns the fully-qualified names of providers such as "registry.terraform.io/hashicorp/aws" to Sentinel. Older versions of Terraform only return the short-form such as "aws".
 
-It was updated on 9/29/2020 to work with both the short name of the AWS provider, "aws", and fully-qualfied provider names that match the regex, `(.*)aws$`. This was required because Terraform 0.13 returns the fully-qualified names of providers such as "registry.terraform.io/hashicorp/aws" to Sentinel. Older versions of Terraform only return the short-form such as "aws".
+It was updated on 2/8/2021 to only look for tags in a given list of AWS
+resources instead of all AWS resources except those in a given list. We made
+this change because we discovered that there are many AWS resources that do
+not include standard AWS tags.
 
 ## Sentinel Module
 This function is contained in the [aws-functions.sentinel](../aws-functions.sentinel) module.
 
 ## Declaration
-`find_resources_with_standard_tags = func()`
+`find_resources_with_standard_tags = func(resource_types)`
 
 ## Arguments
-* None
+* **resource_types**: a list of AWS resource types that should have specified tags defined.
 
 ## Common Functions Used
 None
@@ -26,7 +29,13 @@ This function does not print anything.
 ## Examples
 Here is an example of calling this function, assuming that the aws-functions.sentinel file that contains it has been imported with the alias `aws`:
 ```
-allAWSResourcesWithStandardTags = aws.find_resources_with_standard_tags()
+resource_types = [
+  "aws_s3_bucket",
+  "aws_instance",
+]
+
+allAWSResourcesWithStandardTags =
+                          aws.find_resources_with_standard_tags(resource_types)
 ```
 
 This function is used by the [enforce-mandatory-tags.sentinel](../../enforce-mandatory-tags.sentinel) policy.

governance/third-generation/aws/enforce-mandatory-tags.sentinel

@@ -1,5 +1,5 @@
 # This policy uses the Sentinel tfplan/v2 import to require that
-# all AWS resources that use standard AWS tags have all mandatory tags
+# specified AWS resources have all mandatory tags
 
 # Import common-functions/tfplan-functions/tfplan-functions.sentinel
 # with alias "plan"
@@ -9,18 +9,26 @@ import "tfplan-functions" as plan
 # with alias "aws"
 import "aws-functions" as aws
 
+# List of resources that are required to have name/value tags
+param resource_types default [
+  "aws_s3_bucket",
+  "aws_instance",
+]
+
 # List of mandatory tags
 # Note that the tags here are for internal HashiCorp usage
 # You should assign your own tags in a "mandatory_tags" parameter in your policy set
 # Or change the tags here in the policy.
 param mandatory_tags default ["Name", "ttl", "owner", "se-region", "purpose", "terraform"]
 
 # Get all AWS Resources with standard tags
-allAWSResourcesWithStandardTags = aws.find_resources_with_standard_tags()
+allAWSResourcesWithStandardTags =
+                          aws.find_resources_with_standard_tags(resource_types)
 
 # Filter to AWS resources with violations
 # Warnings will be printed for all violations since the last parameter is true
-violatingAWSResources = plan.filter_attribute_not_contains_list(allAWSResourcesWithStandardTags,
+violatingAWSResources =
+        plan.filter_attribute_not_contains_list(allAWSResourcesWithStandardTags,
                         "tags", mandatory_tags, true)
 
 # Main rule

governance/third-generation/azure/azure-functions/azure-functions.sentinel

@@ -0,0 +1,19 @@
+# Common functions for use with the Azure provider
+
+##### Imports #####
+import "tfplan/v2" as tfplan
+
+##### Functions #####
+
+### find_resources_with_standard_tags ###
+find_resources_with_standard_tags = func(resource_types) {
+  resources = filter tfplan.resource_changes as address, rc {
+    rc.provider_name matches "(.*)azurerm$" and
+    rc.type in resource_types and
+  	rc.mode is "managed" and
+  	(rc.change.actions contains "create" or rc.change.actions contains "update" or
+     rc.change.actions contains "read" or rc.change.actions contains "no-op")
+  }
+
+  return resources
+}

governance/third-generation/azure/azure-functions/docs/find_resources_with_standard_tags.md

@@ -0,0 +1,38 @@
+# find_resources_with_standard_tags
+This function finds all Azure resource instances of specified types in the current plan that are not being permanently deleted using the [tfplan/v2](https://www.terraform.io/docs/cloud/sentinel/import/tfplan-v2.html) import.
+
+This function works with both the short name of the Azure provider, "azurerm", and fully-qualfied provider names that match the regex, `(.*)azurerm$`. The latter is required because Terraform 0.13 and above returns the fully-qualified names of providers such as "registry.terraform.io/hashicorp/azurerm" to Sentinel. Older versions of Terraform only return the short-form such as "azurerm".
+
+## Sentinel Module
+This function is contained in the [azure-functions.sentinel](../azure-functions.sentinel) module.
+
+## Declaration
+`find_resources_with_standard_tags = func(resource_types)`
+
+## Arguments
+* **resource_types**: a list of Azure resource types that should have specified tags defined.
+
+## Common Functions Used
+None
+
+## What It Returns
+This function returns a single flat map of resource instances indexed by the complete [addresses](https://www.terraform.io/docs/internals/resource-addressing.html) of the instances. The map is actually a filtered sub-collection of the [`tfplan.resource_changes`](https://www.terraform.io/docs/cloud/sentinel/import/tfplan-v2.html#the-resource_changes-collection) collection.
+
+## What It Prints
+This function does not print anything.
+
+## Examples
+Here is an example of calling this function, assuming that the aws-functions.sentinel file that contains it has been imported with the alias `azure`:
+```
+resource_types = [
+  "azurerm_resource_group",
+  "azurerm_virtual_machine"
+  "azurerm_linux_virtual_machine",
+  "azurerm_windows_virtual_machine",
+]
+
+allAzureSResourcesWithStandardTags =  
+                        azure.find_resources_with_standard_tags(resource_types)
+```
+
+This function is used by the [enforce-mandatory-tags.sentinel](../../enforce-mandatory-tags.sentinel) policy.

governance/third-generation/azure/enforce-mandatory-tags.sentinel

@@ -1,42 +1,38 @@
 # This policy uses the Sentinel tfplan/v2 import to require that
-# all Azure VMs have all mandatory tags
+# specified Azure resources have all mandatory tags
 
 # Import common-functions/tfplan-functions/tfplan-functions.sentinel
 # with alias "plan"
 import "tfplan-functions" as plan
 
-# List of mandatory tags
-mandatory_tags = ["environment"]
+# Import azure-functions/azure-functions.sentinel
+# with alias "azure"
+import "azure-functions" as azure
 
-# Get all Azure resources using azurerm_virtual_machine
-allAzureVMs = plan.find_resources("azurerm_virtual_machine")
+# List of Azure resources that are required to have name/value tags.
+param resource_types default [
+  "azurerm_resource_group",
+  "azurerm_virtual_machine",
+  "azurerm_linux_virtual_machine",
+  "azurerm_windows_virtual_machine",
+  "azurerm_virtual_network",
+]
 
-# Filter to Azure resources with violations using azurerm_virtual_machine
-# Warnings will be printed for all violations since the last parameter is true
-violatingAzureVMs = plan.filter_attribute_not_contains_list(allAzureVMs,
-                    "tags", mandatory_tags, true)
+# List of mandatory tags
+param mandatory_tags default ["environment"]
 
-# Get all Azure VMs using azurerm_windows_virtual_machine
-allAzureWindowsVMs = plan.find_resources("azurerm_windows_virtual_machine")
+# Get all Azure Resources with standard tags
+allAzureResourcesWithStandardTags =
+                        azure.find_resources_with_standard_tags(resource_types)
 
-# Filter to Azure VMs with violations that use azurerm_windows_virtual_machine
+# Filter to Azure resources with violations using azurerm_virtual_machine
 # Warnings will be printed for all violations since the last parameter is true
-violatingAzureWindowsVMs = plan.filter_attribute_not_contains_list(allAzureWindowsVMs,
+violatingAzureResources =
+      plan.filter_attribute_not_contains_list(allAzureResourcesWithStandardTags,
                     "tags", mandatory_tags, true)
 
-# Get all Azure VMs using azurerm_linux_virtual_machine
-allAzureLinuxVMs = plan.find_resources("azurerm_linux_virtual_machine")
-
-# Filter to Azure VMs with violations that use azurerm_linux_virtual_machine
-# Warnings will be printed for all violations since the last parameter is true
-violatingAzureLinuxVMs = plan.filter_attribute_not_contains_list(allAzureLinuxVMs,
-                    "tags", mandatory_tags, true)
 
 # Main rule
-violations = length(violatingAzureVMs["messages"]) +
-             length(violatingAzureWindowsVMs["messages"]) +
-             length(violatingAzureLinuxVMs["messages"])
-
 main = rule {
-  violations is 0
+  length(violatingAzureResources["messages"]) is 0
 }

governance/third-generation/azure/test/enforce-mandatory-tags/fail.hcl

@@ -1,3 +1,7 @@
+module "azure-functions" {
+  source = "../../azure-functions/azure-functions.sentinel"
+}
+
 module "tfplan-functions" {
   source = "../../../common-functions/tfplan-functions/tfplan-functions.sentinel"
 }

governance/third-generation/azure/test/enforce-mandatory-tags/pass.hcl

@@ -1,3 +1,7 @@
+module "azure-functions" {
+  source = "../../azure-functions/azure-functions.sentinel"
+}
+
 module "tfplan-functions" {
   source = "../../../common-functions/tfplan-functions/tfplan-functions.sentinel"
 }