Add an option to disable hostname override

[mfpierre]

What does this PR do?

Add an option to disable hostname override

Motivation

In EC2 environnement EKS is reporting ec2 internal as node label, this might conflicts in some setup with several cluster where the IP could be the same.

Review checklist

• PR has a meaningful title or PR has the no-changelog label attached
• Feature or bugfix has tests
• Git history is clean
• If PR impacts documentation, docs team has been notified or an issue has been opened on the documentation repo

Additional Notes

Anything else we should know when reviewing?

[2rs2ts]

Hi quick question, let's say I have two nodes, nodeA and nodeB. nodeB is running kube-state-metrics, and nodeA has a datadog-agent running on it that's collecting the metrics provided by kube-state-metrics. Those metrics' host tag will be nodeA if this is true, and nodeB if this is false, right?

[mfpierre]

Hi @2rs2ts,

With the setting to false the hostname will default to the one used by the agent submitting the metric.

For example if you have these prometheus metric exposed on nodeB and we have a running agent on this node collecting KSM metrics.

kube_pod_container_resource_requests_memory_bytes{container="dnsmasq",namespace="kube-system",node="nodeA",pod="kube-dns-1326421443-hj4hx"} 2.097152e+07
kube_pod_container_resource_requests_memory_bytes{container="kubedns",namespace="kube-system",node="nodeB",pod="kube-dns-1326421443-hj4hx"} 2.097152e+07

by default (hostname_override: true) first metric will be attached to host nodeA with the tag node:nodeA.
If set to false the metric will be attached to host nodeB and keeping the tag node:nodeA

Ideally this should always stay true as it's more relevant to have the metric attached to the relevant host, but we've seen issues where the node label exposed by KSM is not unique across multiple clusters (ec2 local-hostname for example) and this is causing the metrics to be attached to a wrong host.

[2rs2ts]

Ok, thanks I understand; I kind of had it backwards and also misunderstood the context. I got part of this explained to me in a support ticket too.

Thanks for the explanation, the example, and of course the fix itself.