Enable Environment.ProcessorCount to return total machine processors in containers

[richlander]

There has been significant discussion and feedback about the value of Environment.ProcessorCount presented in containers. We know that there are scenarios that are underserved by the behavior of this API. We haven't exposed a new API because we haven't developed enough insight on what exactly it would do (as simple as it may seem).

This discussion relates to scenarios where the --cpus or related arguments are used with docker run. There are other ways of controlling CPU usage with docker like CPU affinity that are not related to this proposal.

• Current behavior: Environment.ProcessorCount returns the rounded up decimal value presented via --cpus. For example --cpus 0.4 will result in a return value of 1.
• Proposed OPT-IN behavior: Environment.ProcessorCount returns the value specified by an environment variable.

In terms of the the opt-in mode, for now I'm proposing we expose an environment variable: DOTNET_PROCESSOR_COUNT

The value is converted to an integer and returned as the new value of Environment.ProcessorCount. It must be at > 0 and <= the actual processor count. If those requirements are not met, the actual processor count is returned.

Notes:

• There will be no check that the runtime is running in a container. This new ENV will be generic.
• This value is expected to affect Environment.ProcessorCount for managed code, and any analogues to it in the native runtime code.

Assuming we collect more information on this behavior providing significant benefit, we may provide a new API that provides this information. If you like this idea, please help motivate it with usage information.

We previously (extensively) researched offering the unclipped behavior as the default value for this property. The challenge is that we found that some subsystems always work best when honoring the clipped behavior and other subsystems provide neutral to (potentially much) better behavior when honoring the unclipped behavior. It's this distinction that is the challenge. We have to decide which value each subsystem should honor, and whether we need to expose more configuration options to further tune behavior. We need more findings in the real world to help us on this journey. Provided we have compelling data, multiple avenues are open on this topic.

Related issues:

• Environment.ProcessorCount incorrect reporting in containers in 3.1
• Question: Enviornment.ProcessorCount, Kubernetes/OpenShift and ThreadPool Min Threads
• Add Environment.TotalProcessorCount
• [Breaking change] Environment.ProcessorCount on Windows takes processor affinity into account
• Ambiguity in what Environment.ProcessorCount returns
• Container limits not supported by .NET runtime in process isolation mode -- Windows containers

[dotnet-issue-labeler]

I couldn't figure out the best area label to add to this issue. If you have write-permissions please help me learn by adding exactly one area label.

[stephentoub]

There are lots of places influenced directly or indirectly by the result of ProcessorCount. Is changing it globally for all consumers going to have the desired effect and something someone can reason about? As you said, different subsystems benefit differently from the behaviors. Or is the idea it's just another knob someone can try to see if it helps performance in aggregate?

[richlander]

Great question, @stephentoub. The goal of this is to enable experimentation. For now, you'd get the performance in aggregate, as you call out. Assuming we get adequate and compelling information, we may offer something different.

[stephentoub]

To be clear, I'm fine with us adding the environment variable.

Just trying to understand what that "adequate and compelling information" information would look like. Given that this is a global switch, what would cause us to think we needed to offer something different? Certainly I can imagine the simple case of "I flipped the switch and my perf got better", and we get enough of that signal to decide to change the default (though we already hit issues with that, as you call out). What would feedback look like that would cause us to consider something more fine-grained? Someone flips the switch, gets slightly better perf, takes traces and shares them with us, and that influences our understanding? Something else?

[richlander]

I think there is near zero chance we'll change the default on this property. The experience of doing that last time was bad for everyone involved, and nothing has changed to suggest a different outcome (other than that people would be less tolerant of trying it a second time). I know you probably know that. I'm just wanting to send a clear message to anyone reading this that changing the default isn't on the table.

Certainly I can imagine the simple case of "I flipped the switch and my perf got better", and we get enough of that signal

That's it. If we get interesting information, then we can potentially offer additional customization under a different configuration or something else. We're about to start our container performance testing, soon. I have a doc on that that I intend to publish. As part of that, we should dust off our investigation from #622. We produced a lot of useful data (mostly trapped in email) that suggests that there is "a there, there" with "max procs" enabled. We just didn't get to the point of productizing that, for a variety of reasons (almost entirely schedule problems). This would all lead to exposing one or more different APIs or configuration or something else.

@jkotas was directly involved in those investigations. I'm curious if he has a similar view, that the investigation data suggested that there is a different performance profile enabled by max procs (or at least higher procs than the default) that would be beneficial for some apps. ?? In fairness, I need to re-read those mails to ensure that my memory of that investigation is sound.

[jkotas]

Agree. We know that there are likely some workloads where this setting can be used to optimize performance. This setting will allow us to find out more data about it and unblock the workloads where the default policy works particularly poorly.

This setting can be used as a mitigation for #47427 breaking change.

Certainly I can imagine the simple case of "I flipped the switch and my perf got better"

Yes, we have seen some of that signal e.g. in #11933 (comment) where the container had to be given artificially high quota to make the runtime allow sufficient level of parallelism.

Both Go and Java have equivalent configuration option. It suggests that it is something generally useful.

[AntonLapounov]

The proposal text implies we want to override the value returned by Environment.ProcessorCount without affecting the native runtime, which does not use managed properties. Is that true intention here? I doubt Java's -XX:ActiveProcessorCount works that way. As a side note, Java does not seem to limit the option value by the total number of processors available either.

Does the "actual processor count" in the proposal text refer to the total number of processors available disregarding affinity, cgroup, and other limitations? (Otherwise, we will not be able to mitigate #47427 breaking change.)

DOTNET_ProcessorCount would be a more consistent configuration setting name. We use the underscore in configuration setting names mostly for groups of related settings, e.g., Thread_* or ThreadPool_*.

[jkotas]

The proposal text implies we want to override the value returned by Environment.ProcessorCount without affecting the native runtime

The intention of this proposal is to use the configured count everywhere the runtime uses processor count.

The split between native and managed implementation is accidental and it is changing over time. As you have pointed out, it would not make sense to say that the configuration should be applicable to parts that happen to be implemented in the managed code at the moment.

Java does not seem to limit the option value by the total number of processors available either

I agree that limiting it like this is unnecessary restriction. It makes sense to respect any reasonable positive value, say up to 0xFFFF hex.

DOTNET_ProcessorCount would be a more consistent configuration setting name

Depends what we want to be consistent with. The intentionally designed .NET Core DOTNET_* environment variables are all upper case, a few examples from many DOTNET_ROOT, DOTNET_MULTILEVEL_LOOKUP, DOTNET_SYSTEM_GLOBALIZATION_INVARIANT or DOTNET_SYSTEM_NET_SOCKETS_THREAD_COUNT. The runtime env variables in the CoreCLR configuration system inherited from .NET Framework are typically Pascal case.