[Metricbeat] aws module rds metricset missing cpu usage

[m-standfuss]

After enabling the RDS metricset for the AWS module I see the events coming across but they are missing cpu values (aws.rds.cpu.total.pct). This is for a normal (non-aurora) RDS Postgres db.

ConfigMap for Deployment

...
  aws_rds.yml: |-
    - module: aws
      period: 60s
      metricsets:
        - rds
      tags_filter:
        - key: "Environment"
          value: "<our environment name here>"
...

Resulting ~/modules.d/aws_rds.yml

# cat aws_rds.yml
- module: aws
  period: 60s
  metricsets:
    - rds
  tags_filter:
    - key: "Environment"
      value: "stage"

Example event in metricbeat index:

{
  "_index": "metricbeat-7.17.1-2022.03.17-000004",
  "_type": "_doc",
  "_id": "3RnTnX8BTHDvmoFRS7Jb",
  "_version": 1,
  "_score": 1,
  "_source": {
    "@timestamp": "2022-03-18T16:18:00.000Z",
    "agent": {
      "version": "7.17.1",
      "hostname": "xxxx",
      "ephemeral_id": "xxxx",
      "id": "xxxxx",
      "name": "xxxxx",
      "type": "metricbeat"
    },
    "service": {
      "type": "aws"
    },
    "cloud": {
      "availability_zone": "xxxxx",
      "provider": "aws",
      "region": "xxxx",
      "account": {
        "id": "xxxx",
        "name": "xxxxx"
      }
    },
    "aws": {
      "rds": {
        "read_io": {
          "ops_per_sec": 0
        },
        "throughput": {
          "write": 14472.292128464525,
          "network_receive": 1235.1166666666666,
          "network_transmit": 9321.033333333333,
          "read": 0
        },
        "latency": {
          "read": 0,
          "write": 0.00002
        },
        "replica_lag": {
          "sec": 5
        },
        "database_connections": 0,
        "freeable_memory": {
          "bytes": 266002432
        },
        "disk_queue_depth": 0,
        "write_io": {
          "ops_per_sec": 1.1833136114398093
        },
        "maximum_used_transaction_ids": 2565235,
        "oldest_replication_slot_lag": {
          "mb": -1
        },
        "swap_usage": {
          "bytes": 68968448
        },
        "db_instance": {
          "arn": "xxxxxx",
          "status": "available",
          "identifier": "xxxxx",
          "class": "db.t3.micro",
          "engine_name": "postgres"
        },
        "free_storage": {
          "bytes": 7765127168
        },
        "disk_usage": {
          "replication_slot": {
            "mb": 4096
          },
          "transaction_logs": {
            "mb": 2348818432
          }
        }
      },
      "cloudwatch": {
        "namespace": "AWS/RDS"
      },
      "dimensions": {
        "DBInstanceIdentifier": "xxxxxx"
      },
      "tags": {
        "Name": "xxxxx",
        "Role": "xxxxx",
        "Environment": "xxxxxx"
      }
    },
    "event": {
      "dataset": "aws.rds",
      "module": "aws",
      "duration": 9781510696
    },
    "metricset": {
      "name": "rds",
      "period": 60000
    },
    "ecs": {
      "version": "1.12.0"
    },
    "host": {
      "name": "xxxxx"
    }
  },
  "fields": {
    "aws.rds.oldest_replication_slot_lag.mb": [
      -1
    ],
    "aws.rds.db_instance.status": [
      "available"
    ],
    "aws.rds.disk_usage.transaction_logs.mb": [
      2348818432
    ],
    "cloud.availability_zone": [
      "xxxx"
    ],
    "aws.tags.Role": [
      "xxxxx"
    ],
    "service.type": [
      "aws"
    ],
    "aws.rds.db_instance.identifier": [
      "xxxxxx"
    ],
    "agent.type": [
      "metricbeat"
    ],
    "aws.rds.throughput.write": [
      14472.292
    ],
    "event.module": [
      "aws"
    ],
    "aws.rds.db_instance.arn": [
      "xxxxxx"
    ],
    "aws.dimensions.DBInstanceIdentifier": [
      "xxxxxx"
    ],
    "aws.cloudwatch.namespace": [
      "AWS/RDS"
    ],
    "agent.name": [
      "xxxxx"
    ],
    "host.name": [
      "xxxx"
    ],
    "aws.rds.throughput.network_transmit": [
      9321.033
    ],
    "aws.rds.read_io.ops_per_sec": [
      0
    ],
    "aws.rds.write_io.ops_per_sec": [
      1.1833136
    ],
    "aws.rds.disk_usage.replication_slot.mb": [
      4096
    ],
    "cloud.account.name": [
      "xxxxxx"
    ],
    "cloud.region": [
      "xxxxx"
    ],
    "aws.rds.db_instance.class": [
      "db.t3.micro"
    ],
    "aws.rds.replica_lag.sec": [
      5
    ],
    "aws.tags.Name": [
      "xxxxxx"
    ],
    "metricset.period": [
      60000
    ],
    "aws.rds.maximum_used_transaction_ids": [
      2565235
    ],
    "aws.rds.latency.read": [
      0
    ],
    "aws.rds.free_storage.bytes": [
      7765127168
    ],
    "aws.rds.latency.write": [
      0.00002
    ],
    "agent.hostname": [
      "xxxxxxx"
    ],
    "aws.rds.disk_queue_depth": [
      0
    ],
    "aws.rds.throughput.read": [
      0
    ],
    "aws.rds.throughput.network_receive": [
      1235.1167
    ],
    "metricset.name": [
      "rds"
    ],
    "event.duration": [
      9781510696
    ],
    "aws.tags.Environment": [
      "xxxxxx"
    ],
    "aws.rds.db_instance.engine_name": [
      "postgres"
    ],
    "cloud.provider": [
      "aws"
    ],
    "@timestamp": [
      "2022-03-18T16:18:00.000Z"
    ],
    "agent.id": [
      "2f21e11f-0ce9-428e-b4bd-c72ae341c8b6"
    ],
    "cloud.account.id": [
      "xxxxxx"
    ],
    "ecs.version": [
      "1.12.0"
    ],
    "agent.ephemeral_id": [
      "xxx"
    ],
    "agent.version": [
      "7.17.1"
    ],
    "aws.rds.database_connections": [
      0
    ],
    "aws.rds.swap_usage.bytes": [
      68968448
    ],
    "aws.rds.freeable_memory.bytes": [
      266002432
    ],
    "event.dataset": [
      "aws.rds"
    ]
  }
}

For confirmed bugs, please report:

• Version: image: docker.elastic.co/beats/metricbeat:7.17.1
• Operating System: K8s offical image for container
• Discuss Forum URL:
• Steps to Reproduce: Normal workflow, setup module with provided configuration

[elasticmachine]

Pinging @elastic/integrations (Team:Integrations)

[kaiyan-sheng]

Thanks for reporting this bug. I was able to reproduce it on my side. Also I found out if I use period: 5m instead of 1m, I'm able to collect the CPU usage metric:

[m-standfuss]

Ah thanks I will probably move it to its own schedule and run at 5min for now. Thank you for taking a look at it.

[kaiyan-sheng]

I'm able to reproduce this problem locally with config:

- module: aws
  period: 1m
  credential_profile_name: elastic-beats
  metricsets:
    - cloudwatch
  regions:
    - us-east-1
  metrics:
    - namespace: AWS/RDS
      name: ["CPUUtilization"]
      statistic: ["Average"]
      dimensions:
        - name: DatabaseClass
          value: db.r5.large

I should be getting the CPUUtilization metric every minute for this specific database class db.r5.large but I'm missing a lot of data points.