Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[receiver/kubeletstats] Add feature gate for cpu utilization metrics' deprecation #35139

Merged

Conversation

ChrsMark
Copy link
Member

@ChrsMark ChrsMark commented Sep 11, 2024

Description:

This PR adds a feature gate as discussed at #27885 (comment).
If the feature gate is enabled the container.cpu.utilization, k8s.pod.cpu.utilization and k8s.node.cpu.utilization metrics will be not be disabled being replaced by the container.cpu.usage, k8s.pod.cpu.usage and k8s.node.cpu.usage.

Feature gate schedule

  • alpha: when enabled it makes the .cpu.usage metrics enabled by default
  • beta: .cpu.usage metrics are enabled by default and any configuration enabling the deprecated .cpu.utilization metrics will be failing. Explicitly disabling the feature gate provides the old (deprecated) behavior.
  • stable: .cpu.usage metrics are enabled by default and the deprecated metrics are completely removed.
  • Removed three releases after stable.

@TylerHelmuth @dmitryax let me know what you think about this approach.

Link to tracking Issue:

Testing:

Documentation:

How to test this

  1. Using the following configuration
mode: daemonset
presets:
  kubeletMetrics:
    enabled: true

image:
  repository: otelcontribcol-dev
  tag: "latest"
  pullPolicy: IfNotPresent

command:
  name: otelcontribcol
  extraArgs: [--feature-gates=receiver.kubeletstats.enableCPUUsageMetrics]

config:
  exporters:
    debug:
      verbosity: normal
  receivers:
    kubeletstats:
      collection_interval: 10s
      auth_type: 'serviceAccount'
      endpoint: '${env:K8S_NODE_NAME}:10250'
      insecure_skip_verify: true

  service:
    pipelines:
      metrics:
        receivers: [kubeletstats]
        processors: [batch]
        exporters: [debug]
  1. Ensure that only the .cpu.usage metrics are reported.
  2. Disable the feature gate and check that only the .cpu.utilization metrics are reported.

@ChrsMark ChrsMark force-pushed the kubeletstats_featuregate_metrics branch 4 times, most recently from 55270f1 to 34525a5 Compare September 11, 2024 11:43
@ChrsMark ChrsMark marked this pull request as ready for review September 11, 2024 11:46
@ChrsMark ChrsMark requested a review from a team September 11, 2024 11:46
@ChrsMark ChrsMark force-pushed the kubeletstats_featuregate_metrics branch 2 times, most recently from 858405b to 732836c Compare September 17, 2024 06:56
Comment on lines 296 to 299
Users can use the following metrics instead:
- `k8s.node.cpu.usage`
- `k8s.pod.cpu.usage`
- `container.cpu.usage`
Copy link
Contributor

@povilasv povilasv Sep 17, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would maybe add a note that usage metrics are not utilization metrics. So it's not one to one replacement.

Something like:

Note: usage metrics show usage in terms of resources such as CPU / bytes, not percentage used.
Previous utilization metric values were computed incorrectly.

@ChrsMark ChrsMark force-pushed the kubeletstats_featuregate_metrics branch from 732836c to a699e0b Compare September 17, 2024 12:27
Copy link
Contributor

@povilasv povilasv left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!

receiver/kubeletstatsreceiver/README.md Outdated Show resolved Hide resolved
receiver/kubeletstatsreceiver/README.md Outdated Show resolved Hide resolved
receiver/kubeletstatsreceiver/metadata.yaml Show resolved Hide resolved
receiver/kubeletstatsreceiver/metadata.yaml Show resolved Hide resolved
@ChrsMark ChrsMark force-pushed the kubeletstats_featuregate_metrics branch from a699e0b to 211faf9 Compare September 20, 2024 07:20
@ChrsMark ChrsMark requested a review from a team as a code owner September 20, 2024 07:20
… deprecation

Signed-off-by: ChrsMark <chrismarkou92@gmail.com>
@ChrsMark ChrsMark force-pushed the kubeletstats_featuregate_metrics branch from 25d26e0 to 7aef1b5 Compare September 24, 2024 07:09
Signed-off-by: ChrsMark <chrismarkou92@gmail.com>
@ChrsMark ChrsMark force-pushed the kubeletstats_featuregate_metrics branch from 7aef1b5 to a07194c Compare September 24, 2024 07:18
@ChrsMark
Copy link
Member Author

/cc @dmitryax

Copy link
Member

@dmitryax dmitryax left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@dmitryax dmitryax merged commit 826f2f0 into open-telemetry:main Sep 30, 2024
156 checks passed
@github-actions github-actions bot added this to the next release milestone Sep 30, 2024
jriguera pushed a commit to springernature/opentelemetry-collector-contrib that referenced this pull request Oct 4, 2024
… deprecation (open-telemetry#35139)

**Description:** 

This PR adds a feature gate as discussed at
open-telemetry#27885 (comment).
If the feature gate is enabled the `container.cpu.utilization`,
`k8s.pod.cpu.utilization` and `k8s.node.cpu.utilization` metrics will be
not be disabled being replaced by the `container.cpu.usage`,
`k8s.pod.cpu.usage` and `k8s.node.cpu.usage`.

### Feature gate schedule

- alpha: when enabled it makes the .cpu.usage metrics enabled by default
- beta: .cpu.usage metrics are enabled by default and any configuration
enabling the deprecated .cpu.utilization metrics will be failing.
Explicitly disabling the feature gate provides the old (deprecated)
behavior.
- stable: .cpu.usage metrics are enabled by default and the deprecated
metrics are completely removed.
- Removed three releases after `stable`.

**Documentation:**

### How to test this

1. Using the following configuration
```yaml
mode: daemonset
presets:
  kubeletMetrics:
    enabled: true

image:
  repository: otelcontribcol-dev
  tag: "latest"
  pullPolicy: IfNotPresent

command:
  name: otelcontribcol
  extraArgs: [--feature-gates=receiver.kubeletstats.enableCPUUsageMetrics]

config:
  exporters:
    debug:
      verbosity: normal
  receivers:
    kubeletstats:
      collection_interval: 10s
      auth_type: 'serviceAccount'
      endpoint: '${env:K8S_NODE_NAME}:10250'
      insecure_skip_verify: true

  service:
    pipelines:
      metrics:
        receivers: [kubeletstats]
        processors: [batch]
        exporters: [debug]
```
2. Ensure that only the `.cpu.usage` metrics are reported.
3. Disable the feature gate and check that only the `.cpu.utilization`
metrics are reported.

---------

Signed-off-by: ChrsMark <chrismarkou92@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants