Add process.cpu.count metric

[trask]

The motivation for this PR is to find the right place to capture a metric for Java's Runtime.availableProcessor(), in a way that is language-neutral, rather than throwing it under process.runtime.jvm.*.

See initial attempt at #2384, but as @bogdandrutu pointed out and I confirmed via testing, Runtime.availableProcessor() can be less than the system CPU count, e.g. by launching the process via taskset.

Changes

Adds a new metric process.cpu.count to capture the number of CPUs available to the process.

Two open questions:

• type: Gauge vs Async UpDownCounter
• name: process.cpu.count vs process.cpu.available

My initial thoughts on these questions:

• type: Async UpDownCounter since it's counting # of CPUs.
• name: process.cpu.count seems simpler, and available sounds like availability, but I don't have strong feelings about this.

[trask]

@tigrannajaryan @bogdandrutu related to #2384, is it ok for Java process itself to report process.cpu.time and process.cpu.count? or should we define new metrics under process.runtime.jvm.* if we want to report those from inside the Java process?

[bogdandrutu]

@trask would be good to see if we have more cases like this. For example in a k8s environment, do we consider the cpu_limit as being equivalent with this? Are there any other languages with similar capability?

[trask]

@trask would be good to see if we have more cases like this

would it be better for us to define process.runtime.jvm.cpu.time and process.runtime.jvm.cpu.count for now then? we can always migrate to process.cpu.time and process.cpu.count in the future via schema mapping

[bogdandrutu]

If we cannot find any other use-cases the jvm versions are the answer, but was looking to see if other maintainers have some input here.

[trask]

looking to see if other maintainers have some input here

@open-telemetry/dotnet-approvers @open-telemetry/go-approvers do the .NET or Go runtimes have the equivalent of process.cpu.time (and maybe process.cpu.count)?

e.g. similar to the JVM's

• OperatingSystemMXBean#getProcessCpuTime()
• Runtime#availableProcessors()

[pellared]

looking to see if other maintainers have some input here

@open-telemetry/dotnet-approvers @open-telemetry/go-approvers do the .NET or Go runtimes have the equivalent of process.cpu.time (and maybe process.cpu.count)?

e.g. similar to the JVM's

• OperatingSystemMXBean#getProcessCpuTime()
• Runtime#availableProcessors()

.NET:

• process.cpu.time - could be done e.g. via DateTime.UtcNow - Process.GetCurrentProcess().StartTime.ToUniversalTime()
• process.cpu.count - Environment.ProcessorCount

Go:

• process.cpu.time - AFAIK nothing OOTB. But something like cpu.Times from https://github.com/shirou/gopsutil could be implemented/used. or some time.Now in initialized in a global variable and calculating the interval using time.Since if the precision is not very important
• process.cpu.count - runtime.NumCPU

[carlosalberto]

From what @pellared posted, DotNet says that

The value returned by this API is fixed at .NET runtime startup for the process lifetime. It does not reflect changes in the environment settings while the process is running.

And for Go:

The set of available CPUs is checked by querying the operating system at process startup. Changes to operating system CPU allocation after process startup are not reflected.

Which means, at least these specific values, cannot be used for metrics purpose, as they don't change.

[Aneurysm9]

Go's host instrumentation does have process.cpu.time, which is obtained from the gopsutil/process package @pellared mentioned.

[trask]

@bogdandrutu do you need more maintainer feedback?

It looks like Go is already capturing process.cpu.time, and .NET is discussing that they would like to as well: open-telemetry/opentelemetry-dotnet-contrib#207 (comment)

I'd like to propose that we move forward with this PR, and with allowing language-based instrumentation to emit process.* metrics for their own process.

[carlosalberto]

Ping @bogdandrutu

[carlosalberto]

Hey @trask -

Bogdan is on holidays this week. Shall we come back to the discussion next week?

[trask]

no problem

[carlosalberto]

Ping @bogdandrutu

[reyang]

With mobile devices moving towards to the asymmetric multi-core processor, do we consider a dimension which could tell whether it is a Performance vs. Efficiency core? Probably not an interesting topic for service developers, but definitely interesting for device/mobile.

[bogdandrutu]

I think it is fine to report a metric like this, but was curios if we need count, or "utilization" to be consistent with system.

[bogdandrutu]

For consistency with system metrics, should we report proess.cpu.utilization instead? See https://github.com/open-telemetry/opentelemetry-specification/blob/main/specification/metrics/semantic_conventions/system-metrics.md#metric-instruments

[trask]

I agree that utilization is what users generally want to see. But utilization collected client-side is just a gauge and can't be aggregated over time, so I think it's nice to capture process.cpu.time and process.cpu.count where possible and display utilization based on those two metrics.

[bogdandrutu]

We had the same debate for system.cpu.utilization and the result of the discussion is that system.cpu.utilization was simpler for some backends than calculating from two metrics.

But utilization collected client-side is just a gauge and can't be aggregated over time

Not sure this is true, since if you calculate "delta utilization" if process.cpu.count does not change, you can correctly merge them by doing sum:

Timestamp 0 -> cpu.time0/count
Timestamp 1 -> cpu.time1/count -> report (cpu.time 1 - cpu.time 0) / count / (Timestamp 1 - Timestamp 0)
Timestamp 2 -> cpu.time1/count -> report (cpu.time 2 - cpu.time 1) / count / (Timestamp 2 - Timestamp 1) -> percent of total cpu / second.

The idea was that if we report roughly every same interval (also we have start and current time) you can average them two reported value to get the utilization over (Timestamp 2 - Timestamp 0).

If the argument is wrong for the system.cpu.utilization we should change that as well, I want consistency :)

[trask]

Interesting. I don't have any objection to that approach. From a JVM metrics perspective, there was some interest in capturing available cpu count separate from utilization because it gives clues about GC and common thread pool sizing, but we haven't mapped out GC or thread pool metrics yet, so will revisit that then if/when we have a more specific need. I'll send a new PR to propose adding process.cpu.utilization, with this definition of process.cpu.time divided by elapsed time divided by "available processor count"

[trask]

Based on discussion with @bogdandrutu above, I have created #2436 instead.

Closing this, will revisit if/when we have a specific need for it.