[Prometheus Receiver] Generic OTel Prometheus Receiver configuration for each K8s component that works for all or major certified K8s distros

[developer1622]

Component(s)

receiver/prometheus

Is your feature request related to a problem? Please describe.

I don't know if there was something already exists.

OTel has kubeletstatsreceiver and k8sclusterreceiver, which fetches only some K8s component metrics.

I was wondering if there is a generic or template OpenTelemetry configuration manifest that we can apply to any or major certified K8s distribution such that I would be able to get every K8s component Prometheus metrics (both control plane components and data plane components), assuming that OpenTelemetry could be deployed as Daemonset(for Kubeproxy etc.) and Deployments (etcd, coredns, apiserver, kube-api scheduler, kube-controller etc ).

The major problem is writing scrape configuration for each variant of certified K8s distribution, which might vary with their labels and annotations.

Describe the solution you'd like

I have tried some scrape configurations for the vanilla K8s cluster. Some of the components worked, but not all.

I did try something from the Kubecon event:

https://raw.githubusercontent.com/pavolloffay/kubecon-na-2023-opentelemetry-kubernetes-metrics-tutorial/main/backend/06-collector-k8s-cluster-metrics.yaml

As I said, it is not working for some components and I will have to change the scrape configuration suitable for the running K8s cluster. Some of the components worked, but not all

Describe alternatives you've considered

I am not sure and I did not consider any idea.

Additional context

A single generic standard OTel Prometheus receiver configuration that fetches the metrics of all the K8s cluster components (control plane and data plane) for major or any certified K8s distribution when deployed as both Deamonset and Deployment.

We need two OTel Prometheus receiver configurations: one for Deployment(apiserver, etcd, coredns, cloud controller(if on a cloud) etc.) and another for Kube proxy, CNI, and CSI(if they are exposed).

Kubelet has been covered under OTel native kubeletstats receiver.

Thank you.

[github-actions]

Pinging code owners:

• receiver/prometheus: @Aneurysm9 @dashpole

See Adding Labels via Comments if you do not have permissions to add labels yourself.

[developer1622]

Thank you @dashpole for adding labels

May I know if there has been any discussion happened on this? Thanks

[dashpole]

I don't know of an example like that today, but anyone is welcome to contribute one. There is a logging example for k8s: https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/examples/kubernetes. It might be good to add the metrics example separately.

Otherwise, this is waiting for someone interested in working on it.

[developer1622]

Hi @dashpole, thank you for your response. I appreciate your help with confirmation.