in k8s prometheus receiver does not have the ability to only scrape the metrics of pods on its own node

[yutingcaicyt]

Component(s)

receiver/prometheus

Is your feature request related to a problem? Please describe.

The collector is deployed by deamonset in my project. I want to collect metrics of apps in each pod with the Prometheus receiver, but I found the receiver can't only scrape the metrics of pods on its own node. Is there any method to solve this problem?

Describe the solution you'd like

Prometheus receiver can scrape the metrics of pods on its own node.

Describe alternatives you've considered

No response

Additional context

No response

[github-actions]

Pinging code owners:

• receiver/prometheus: @Aneurysm9 @dashpole

See Adding Labels via Comments if you do not have permissions to add labels yourself.

[jinja2]

You can use the receivercreator with the k8s_observer for this. Refer to the prometheus receiver example here.

[dashpole]

You can also use the field selector in the prometheus scrape config:

  scrape_configs:
    - job_name: kubernetes_pods
      kubernetes_sd_configs:
        - role: pod
          selectors:
            - role: pod
              field: spec.nodeName=$NODE_NAME

Then on your pod spec:

  env:
  - name: NODE_NAME
    valueFrom:
      fieldRef:
        fieldPath: spec.nodeName

[yutingcaicyt]

But in my understanding, selectors.field does not recognize environment variables. Will this really work? @dashpole

[dashpole]

The collector does env var substitution. So $NODE_NAME is passed to the prometheus receiver config as the value of the env var, not as the "$NODE_NAME" string. This works in the otel collector, but does not work in upstream prometheus server. I've used this quite extensively myself.

[github-actions]

This issue has been inactive for 60 days. It will be closed in 60 days if there is no activity. To ping code owners by adding a component label, see Adding Labels via Comments, or if you are unsure of which component this issue relates to, please ping @open-telemetry/collector-contrib-triagers. If this issue is still relevant, please ping the code owners or leave a comment explaining why it is still relevant. Otherwise, please close it.

[dashpole]

working as intended.