[processor/k8sattributes] Retrieve pods at the namespace scope

[edenkoveshi]

Is your feature request related to a problem? Please describe.

I want to extract attributes from pods in a single namespace, and I don't really need a cluster role binding for that
What I want is to extract pods attributes with a namespace RoleBinding
Should be quite easy
I can help implementing this if needed

Describe the solution you'd like

Change the Kubernetes client to fetch items at the namespace level if namespace is defined

Describe alternatives you've considered

No response

Additional context

No response

[github-actions]

Pinging code owners: @owais @dmitryax. See Adding Labels via Comments if you do not have permissions to add labels yourself.

[gouslu]

Hi @edenkoveshi, I have a few questions if you don't mind:

• What attributes do you want to extract?
• Do you mean you want to extract "attributes" of a pod within a specific namespace, a namespace name being "RoleBinding"? I am not sure if I got that.
• What do you mean by not needing a cluster role binding, why would you need a role binding in the first place?

I appreciate if you can elaborate a little bit. I am recently getting into OpenTelemetry, and would like to start contributing, it would help me if you can explain me what you need with more details.

[edenkoveshi]

From the documentation
The processor uses the kubernetes API to discover all pods running in a cluster, keeps a record of their IP addresses, pod UIDs and interesting metadata.
It queries the API for pods within the entire cluster, which requires ClusterRoleBinding

I want to achieve the same thing (I only want some labels actually but it doesn't really matter) but not for pods in the entire cluster, only from selected namespaces, and use namespaced RoleBindings to avoid over-priveliging

Edit: This really can be summed up to changing the API call to include namespaces
I wanted to know if there's any interest in this

[gouslu]

@edenkoveshi thanks, this explains everything I needed to understand. I will fiddle with it and will let you know if I end up creating a PR to address this (unless someone who is already member picks it up before that). Cheers!

[fatsheep9146]

I think this is also block by #15651, we should solve that first. cc @dmitryax

[github-actions]

This issue has been inactive for 60 days. It will be closed in 60 days if there is no activity. To ping code owners by adding a component label, see Adding Labels via Comments, or if you are unsure of which component this issue relates to, please ping @open-telemetry/collector-contrib-triagers. If this issue is still relevant, please ping the code owners or leave a comment explaining why it is still relevant. Otherwise, please close it.

Pinging code owners:

• processor/k8sattributes: @dmitryax @rmfitzpatrick

See Adding Labels via Comments if you do not have permissions to add labels yourself.

[fatsheep9146]

I think is issue is ready to be implemented, since k8sattributeprocessor e2e is enabled?
@dmitryax

[dmitryax]

Yes, I think it should be good to go with this enhancement

[github-actions]

This issue has been inactive for 60 days. It will be closed in 60 days if there is no activity. To ping code owners by adding a component label, see Adding Labels via Comments, or if you are unsure of which component this issue relates to, please ping @open-telemetry/collector-contrib-triagers. If this issue is still relevant, please ping the code owners or leave a comment explaining why it is still relevant. Otherwise, please close it.

Pinging code owners:

• processor/k8sattributes: @dmitryax @rmfitzpatrick

See Adding Labels via Comments if you do not have permissions to add labels yourself.

[github-actions]

This issue has been inactive for 60 days. It will be closed in 60 days if there is no activity. To ping code owners by adding a component label, see Adding Labels via Comments, or if you are unsure of which component this issue relates to, please ping @open-telemetry/collector-contrib-triagers. If this issue is still relevant, please ping the code owners or leave a comment explaining why it is still relevant. Otherwise, please close it.

Pinging code owners:

• processor/k8sattributes: @dmitryax @rmfitzpatrick @fatsheep9146

See Adding Labels via Comments if you do not have permissions to add labels yourself.

[pgoringe]

This feature would be benefical to our multi-tenant k8s cluster as it provides segregation between namespaces. It would enable each tenant to deploy an OpenTelemetry Collector in their own namespace without the need for cluster scoped RBAC (allowing the service account access to other namespaces). I hope it can be added. Thanks!

[dmitryax]

@pgoringe, thanks for your comment. @edenkoveshi @pgoringe @fatsheep9146, are any of you interested in implementing this feature? Any help is appreciated.

[github-actions]

This issue has been inactive for 60 days. It will be closed in 60 days if there is no activity. To ping code owners by adding a component label, see Adding Labels via Comments, or if you are unsure of which component this issue relates to, please ping @open-telemetry/collector-contrib-triagers. If this issue is still relevant, please ping the code owners or leave a comment explaining why it is still relevant. Otherwise, please close it.

Pinging code owners:

• processor/k8sattributes: @dmitryax @rmfitzpatrick @fatsheep9146 @TylerHelmuth

See Adding Labels via Comments if you do not have permissions to add labels yourself.

[Exactol]

I was able to run the Collector within a namespace on an older release (v0.74.0), but after upgrading to the latest version I'm seeing errors in the log about: "failed to list *v1.Namespace: namespaces "kube-system" is forbidden. User "***" cannot list resource "namespaces" in API group "" at the cluster scope

I think there is a regression in v0.83.0, possibly caused by the change to add k8s.cluster.uid

[jinja2]

Yeah, the k8s.cluster.uid change introduces namespace lookup even when the attribute is not enabled (it is disabled by default). I think the processor will still work but there'll be a bunch of errors complaining about not able to list namespace. Let me try a PR to fix this.

Regarding the original ask, once the aforementioned issue is fixed, users can indeed use the processor with Rolebinding to extract labels/annotations for pods running in the collector's namespace only. You need to set the namespace filter for this.

      k8sattributes:
        filter:
          namespace: COLLECTOR_NAMESPACE

One caveat here, even with the namespace filter set, the processor currently cannot extract metadata from the namespace (doesn't seem to pass the filter when ns is queried). I think the issue should be updated to fix this. Ideally, the processor when run with the filter.namespace set to the namespace it is running in, should be able to handle extracting labels/annotations from the namespace and pods within the ns without needing any cluster-scoped access.

[github-actions]

This issue has been inactive for 60 days. It will be closed in 60 days if there is no activity. To ping code owners by adding a component label, see Adding Labels via Comments, or if you are unsure of which component this issue relates to, please ping @open-telemetry/collector-contrib-triagers. If this issue is still relevant, please ping the code owners or leave a comment explaining why it is still relevant. Otherwise, please close it.

Pinging code owners:

• processor/k8sattributes: @dmitryax @rmfitzpatrick @fatsheep9146 @TylerHelmuth

See Adding Labels via Comments if you do not have permissions to add labels yourself.

[hkailantzis]

any news on this ?, would be great, as we're facing the same issue, (cluster-scoped access). thanks! cc @jinja2 @dmitryax @rmfitzpatrick @fatsheep9146 @TylerHelmuth

[jinja2]

@hkailantzis this one fell off my radar, but I should be able to look into this next week

[samuellvicente]

Adding interest in this, I have the same issue where my cluster is shared between teams and we use namespaces to isolate things, it would be great if we could scope the processor to just a namespace.

[hkailantzis]

Hi @jinja2 , are there any updates on this ? Is there something else blocking it ? Thanks in advance. :).

[jinja2]

There is a PR open, hopefully it'll make it in in 0.97 release.

[atoulme]

OK to close as the PR is now merged?