Deprecate tags.resource_type_filter config parameter and replace with…

… resource_type (elastic#19688)

* Deprecate tags.resource_type_filter config parameter

CHANGELOG.next.asciidoc

@@ -573,6 +573,7 @@ https://github.com/elastic/beats/compare/v7.0.0-alpha2...master[Check the HEAD d
 
 *Metricbeat*
 - Deprecate tags config parameter in cloudwatch metricset. {pull}16733[16733]
+- Deprecate tags.resource_type_filter config parameter and replace with resource_type. {pull}19688[19688]
 
 *Packetbeat*
 

metricbeat/docs/modules/aws.asciidoc

@@ -315,7 +315,7 @@ metricbeat.modules:
           value: i-0686946e22cf9494a
     - namespace: AWS/EBS
     - namespace: AWS/ELB
-      tags.resource_type_filter: elasticloadbalancing
+      resource_type: elasticloadbalancing
       tags:
         - key: "Organization"
           value: "Engineering"

x-pack/elastic-agent/docs/elastic-agent_configuration_example.yml

@@ -350,7 +350,7 @@ inputs:
       dataset.name: aws.cloudwatch
       period: 5m
       name: ["CPUUtilization", "DiskWriteOps"]
-      tags.resource_type_filter: ec2:instance
+      resource_type: ec2:instance
       #dimensions:
       #  - name: InstanceId
       #    value: i-0686946e22cf9494a

x-pack/metricbeat/metricbeat.reference.yml

@@ -219,7 +219,7 @@ metricbeat.modules:
           value: i-0686946e22cf9494a
     - namespace: AWS/EBS
     - namespace: AWS/ELB
-      tags.resource_type_filter: elasticloadbalancing
+      resource_type: elasticloadbalancing
       tags:
         - key: "Organization"
           value: "Engineering"

x-pack/metricbeat/module/aws/_meta/config.epr.yml

@@ -79,7 +79,7 @@
   metrics:
     - namespace: AWS/EC2
       name: [""]
-      tags.resource_type_filter: ""
+      resource_type: ""
       dimensions:
         - name: ""
           value: ""

x-pack/metricbeat/module/aws/_meta/config.reference.yml

@@ -34,7 +34,7 @@
           value: i-0686946e22cf9494a
     - namespace: AWS/EBS
     - namespace: AWS/ELB
-      tags.resource_type_filter: elasticloadbalancing
+      resource_type: elasticloadbalancing
       tags:
         - key: "Organization"
           value: "Engineering"

x-pack/metricbeat/module/aws/_meta/config.yml

@@ -14,7 +14,7 @@
   metrics:
     - namespace: AWS/EC2
       #name: ["CPUUtilization", "DiskWriteOps"]
-      tags.resource_type_filter: ec2:instance
+      resource_type: ec2:instance
       #dimensions:
       #  - name: InstanceId
       #    value: i-0686946e22cf9494a

x-pack/metricbeat/module/aws/cloudwatch/_meta/docs.asciidoc

@@ -20,9 +20,8 @@ iam:ListAccountAliases
 For example, AWS/EC2, AWS/S3. If wildcard * is given for namespace, metrics
 from all namespaces will be collected automatically.
 * *name*: The name of the metric to filter against. For example, CPUUtilization for EC2 instance.
-* *dimensions*: The dimensions to filter against. For example, InstanceId=i-123. Dimension value
-could be wildcard `*` to match any value.
-* *tags.resource_type_filter*: The constraints on the resources that you want returned.
+* *dimensions*: The dimensions to filter against. For example, InstanceId=i-123.
+* *resource_type*: The constraints on the resources that you want returned.
 The format of each resource type is service[:resourceType].
 For example, specifying a resource type of ec2 returns all Amazon EC2 resources
 (which includes EC2 instances). Specifying a resource type of ec2:instance returns
@@ -53,7 +52,7 @@ in configurations in order for this metricset to make proper AWS API calls.
   metrics:
     - namespace: AWS/EBS <1>
     - namespace: AWS/ELB <2>
-      tags.resource_type_filter: elasticloadbalancing
+      resource_type: elasticloadbalancing
       tags:
         - key: "Organization"
           value: "Engineering"
@@ -69,9 +68,9 @@ in configurations in order for this metricset to make proper AWS API calls.
 specific namespace, such as `AWS/EBS`.
 
 <2> `cloudwatch` metricset also has the ability to collect tags from AWS resources.
-If user specify `tags.resource_type_filter`, then tags will be collected and stored
+If user specify `resource_type`, then tags will be collected and stored
 as a part of the event. Please see https://docs.aws.amazon.com/resourcegroupstagging/latest/APIReference/API_GetResources.html[AWS API GetResources]
-for more details about `tags.resource_type_filter`.
+for more details about `resource_type`.
 
 <3> If users knows exactly what are the cloudwatch metrics they want to collect,
 this configuration format can be used. `namespace` and `metricname` need to be
@@ -128,11 +127,11 @@ from EBS, ELB and EC2 with tags from these services.
     - cloudwatch
   metrics:
     - namespace: AWS/EBS
-      tags.resource_type_filter: ebs
+      resource_type: ebs
     - namespace: AWS/ELB
-      tags.resource_type_filter: elasticloadbalancing
+      resource_type: elasticloadbalancing
     - namespace: AWS/EC2
-      tags.resource_type_filter: ec2:instance
+      resource_type: ec2:instance
 ----
 
 With the configuration below, users will be able to collect specific cloudwatch
@@ -147,7 +146,7 @@ metric(average) from EC2 instance i-456.
   metrics:
     - namespace: AWS/EC2
       name: ["CPUUtilization"]
-      tags.resource_type_filter: ec2:instance
+      resource_type: ec2:instance
       dimensions:
         - name: InstanceId
           value: i-123

x-pack/metricbeat/module/aws/cloudwatch/cloudwatch.go

@@ -68,7 +68,8 @@ type Config struct {
 	Namespace          string      `config:"namespace" validate:"nonzero,required"`
 	MetricName         []string    `config:"name"`
 	Dimensions         []Dimension `config:"dimensions"`
-	ResourceTypeFilter string      `config:"tags.resource_type_filter"`
+	ResourceTypeFilter string      `config:"tags.resource_type_filter"` // Deprecated.
+	ResourceType       string      `config:"resource_type"`
 	Statistic          []string    `config:"statistic"`
 	Tags               []aws.Tag   `config:"tags"` // Deprecated.
 }
@@ -78,6 +79,10 @@ func (c Config) Validate() error {
 	if c.Tags != nil {
 		cfgwarn.Deprecate("8.0.0", "tags is deprecated. Use tags_filter instead")
 	}
+
+	if c.ResourceTypeFilter != "" {
+		cfgwarn.Deprecate("8.0.0", "tags.resource_type_filter is deprecated. Use resource_type instead")
+	}
 	return nil
 }
 
@@ -350,11 +355,11 @@ func (m *MetricSet) readCloudwatchConfig() (listMetricWithDetail, map[string][]n
 				metricsWithStatsTotal = append(metricsWithStatsTotal, metricsWithStats)
 			}
 
-			if config.ResourceTypeFilter != "" {
-				if _, ok := resourceTypesWithTags[config.ResourceTypeFilter]; ok {
-					resourceTypesWithTags[config.ResourceTypeFilter] = tagsFilter
+			if config.ResourceType != "" {
+				if _, ok := resourceTypesWithTags[config.ResourceType]; ok {
+					resourceTypesWithTags[config.ResourceType] = tagsFilter
 				} else {
-					resourceTypesWithTags[config.ResourceTypeFilter] = append(resourceTypesWithTags[config.ResourceTypeFilter], tagsFilter...)
+					resourceTypesWithTags[config.ResourceType] = append(resourceTypesWithTags[config.ResourceType], tagsFilter...)
 				}
 			}
 			continue
@@ -364,7 +369,7 @@ func (m *MetricSet) readCloudwatchConfig() (listMetricWithDetail, map[string][]n
 			names:              config.MetricName,
 			tags:               tagsFilter,
 			statistics:         config.Statistic,
-			resourceTypeFilter: config.ResourceTypeFilter,
+			resourceTypeFilter: config.ResourceType,
 			dimensions:         cloudwatchDimensions,
 		}
 
@@ -494,7 +499,7 @@ func (m *MetricSet) createEvents(svcCloudwatch cloudwatchiface.ClientAPI, svcRes
 	// Find a timestamp for all metrics in output
 	timestamp := aws.FindTimestamp(metricDataResults)
 
-	// Create events when there is no tags_filter or tags.resource_type_filter specified.
+	// Create events when there is no tags_filter or resource_type specified.
 	if len(resourceTypeTagFilters) == 0 {
 		if !timestamp.IsZero() {
 			for _, output := range metricDataResults {

x-pack/metricbeat/module/aws/cloudwatch/cloudwatch_test.go

@@ -446,8 +446,8 @@ func TestReadCloudwatchConfig(t *testing.T) {
 							Value: "i-1",
 						},
 					},
-					ResourceTypeFilter: "ec2:instance",
-					Statistic:          []string{"Average"},
+					ResourceType: "ec2:instance",
+					Statistic:    []string{"Average"},
 				},
 			},
 			nil,
@@ -466,8 +466,8 @@ func TestReadCloudwatchConfig(t *testing.T) {
 							Value: "i-1",
 						},
 					},
-					ResourceTypeFilter: "ec2:instance",
-					Statistic:          []string{"Average"},
+					ResourceType: "ec2:instance",
+					Statistic:    []string{"Average"},
 				},
 				{
 					Namespace: "AWS/S3",
@@ -489,8 +489,8 @@ func TestReadCloudwatchConfig(t *testing.T) {
 							Value: "i-1",
 						},
 					},
-					Statistic:          []string{"Average"},
-					ResourceTypeFilter: "ec2:instance",
+					Statistic:    []string{"Average"},
+					ResourceType: "ec2:instance",
 				},
 				{
 					Namespace: "AWS/Lambda",
@@ -508,8 +508,8 @@ func TestReadCloudwatchConfig(t *testing.T) {
 							Value: "READER",
 						},
 					},
-					Statistic:          []string{"Average"},
-					ResourceTypeFilter: "rds",
+					Statistic:    []string{"Average"},
+					ResourceType: "rds",
 				},
 			},
 			nil,
@@ -520,13 +520,13 @@ func TestReadCloudwatchConfig(t *testing.T) {
 			"Test a specific metric (only with metric name) and a namespace",
 			[]Config{
 				{
-					Namespace:          "AWS/EC2",
-					MetricName:         []string{"CPUUtilization"},
-					ResourceTypeFilter: resourceTypeEC2,
+					Namespace:    "AWS/EC2",
+					MetricName:   []string{"CPUUtilization"},
+					ResourceType: resourceTypeEC2,
 				},
 				{
-					Namespace:          "AWS/S3",
-					ResourceTypeFilter: "s3",
+					Namespace:    "AWS/S3",
+					ResourceType: "s3",
 				},
 			},
 			nil,
@@ -539,8 +539,8 @@ func TestReadCloudwatchConfig(t *testing.T) {
 			"test EBS namespace",
 			[]Config{
 				{
-					Namespace:          "AWS/EBS",
-					ResourceTypeFilter: "ec2",
+					Namespace:    "AWS/EBS",
+					ResourceType: "ec2",
 				},
 			},
 			nil,
@@ -553,10 +553,10 @@ func TestReadCloudwatchConfig(t *testing.T) {
 			"test with two metrics and no dimension",
 			[]Config{
 				{
-					Namespace:          "AWS/EC2",
-					MetricName:         []string{"CPUUtilization", "StatusCheckFailed"},
-					ResourceTypeFilter: resourceTypeEC2,
-					Statistic:          []string{"Average", "Maximum"},
+					Namespace:    "AWS/EC2",
+					MetricName:   []string{"CPUUtilization", "StatusCheckFailed"},
+					ResourceType: resourceTypeEC2,
+					Statistic:    []string{"Average", "Maximum"},
 				},
 			},
 			nil,
@@ -583,21 +583,21 @@ func TestReadCloudwatchConfig(t *testing.T) {
 			"Test with different statistics",
 			[]Config{
 				{
-					Namespace:          "AWS/EC2",
-					MetricName:         []string{"CPUUtilization"},
-					ResourceTypeFilter: resourceTypeEC2,
+					Namespace:    "AWS/EC2",
+					MetricName:   []string{"CPUUtilization"},
+					ResourceType: resourceTypeEC2,
 				},
 				{
-					Namespace:          "AWS/ELB",
-					MetricName:         []string{"BackendConnectionErrors", "HTTPCode_Backend_2XX", "HTTPCode_Backend_3XX"},
-					Statistic:          []string{"Sum"},
-					ResourceTypeFilter: "elasticloadbalancing",
+					Namespace:    "AWS/ELB",
+					MetricName:   []string{"BackendConnectionErrors", "HTTPCode_Backend_2XX", "HTTPCode_Backend_3XX"},
+					Statistic:    []string{"Sum"},
+					ResourceType: "elasticloadbalancing",
 				},
 				{
-					Namespace:          "AWS/ELB",
-					MetricName:         []string{"HealthyHostCount", "SurgeQueueLength", "UnHealthyHostCount"},
-					Statistic:          []string{"Maximum"},
-					ResourceTypeFilter: "elasticloadbalancing",
+					Namespace:    "AWS/ELB",
+					MetricName:   []string{"HealthyHostCount", "SurgeQueueLength", "UnHealthyHostCount"},
+					Statistic:    []string{"Maximum"},
+					ResourceType: "elasticloadbalancing",
 				},
 			},
 			[]aws.Tag{
@@ -615,16 +615,16 @@ func TestReadCloudwatchConfig(t *testing.T) {
 			"Test with different statistics and a specific metric",
 			[]Config{
 				{
-					Namespace:          "AWS/ELB",
-					MetricName:         []string{"BackendConnectionErrors", "HTTPCode_Backend_2XX", "HTTPCode_Backend_3XX"},
-					Statistic:          []string{"Sum"},
-					ResourceTypeFilter: "elasticloadbalancing",
+					Namespace:    "AWS/ELB",
+					MetricName:   []string{"BackendConnectionErrors", "HTTPCode_Backend_2XX", "HTTPCode_Backend_3XX"},
+					Statistic:    []string{"Sum"},
+					ResourceType: "elasticloadbalancing",
 				},
 				{
-					Namespace:          "AWS/ELB",
-					MetricName:         []string{"HealthyHostCount", "SurgeQueueLength", "UnHealthyHostCount"},
-					Statistic:          []string{"Maximum"},
-					ResourceTypeFilter: "elasticloadbalancing",
+					Namespace:    "AWS/ELB",
+					MetricName:   []string{"HealthyHostCount", "SurgeQueueLength", "UnHealthyHostCount"},
+					Statistic:    []string{"Maximum"},
+					ResourceType: "elasticloadbalancing",
 				},
 				{
 					Namespace: "AWS/Lambda",
@@ -638,8 +638,8 @@ func TestReadCloudwatchConfig(t *testing.T) {
 							Value: "i-1",
 						},
 					},
-					Statistic:          []string{"Average"},
-					ResourceTypeFilter: "ec2:instance",
+					Statistic:    []string{"Average"},
+					ResourceType: "ec2:instance",
 				},
 				{
 					Namespace:  "AWS/RDS",
@@ -654,8 +654,8 @@ func TestReadCloudwatchConfig(t *testing.T) {
 							Value: "READER",
 						},
 					},
-					Statistic:          []string{"Average"},
-					ResourceTypeFilter: "rds",
+					Statistic:    []string{"Average"},
+					ResourceType: "rds",
 				},
 			},
 			[]aws.Tag{
@@ -678,8 +678,8 @@ func TestReadCloudwatchConfig(t *testing.T) {
 							Value: "i-1",
 						},
 					},
-					Statistic:          []string{"Average"},
-					ResourceTypeFilter: "ec2:instance",
+					Statistic:    []string{"Average"},
+					ResourceType: "ec2:instance",
 				},
 			},
 			nil,
@@ -700,8 +700,8 @@ func TestReadCloudwatchConfig(t *testing.T) {
 							Value: "i-1",
 						},
 					},
-					ResourceTypeFilter: "ec2:instance",
-					Statistic:          []string{"Average"},
+					ResourceType: "ec2:instance",
+					Statistic:    []string{"Average"},
 				},
 			},
 			nil,

x-pack/metricbeat/module/aws/dynamodb/manifest.yml

@@ -5,7 +5,7 @@ input:
   defaults:
     metrics:
       - namespace: AWS/DynamoDB
-        tags.resource_type_filter: dynamodb
+        resource_type: dynamodb
         statistic: ["Average"]
         name:
           - SuccessfulRequestLatency
@@ -19,7 +19,7 @@ input:
           - AccountProvisionedReadCapacityUtilization
           - AccountProvisionedWriteCapacityUtilization
       - namespace: AWS/DynamoDB
-        tags.resource_type_filter: dynamodb
+        resource_type: dynamodb
         statistic: ["Sum"]
         name:
           - SystemErrors
@@ -32,7 +32,7 @@ input:
           - ThrottledRequests
           - WriteThrottleEvents
       - namespace: AWS/DynamoDB
-        tags.resource_type_filter: dynamodb
+        resource_type: dynamodb
         statistic: ["Maximum"]
         name:
           - SuccessfulRequestLatency