[release/6.0] Harden native gate thread against affinity mask growth #59132
Conversation
Hardened ThreadpoolMgr::GateThreadStart against the possibility that the observed group-local affinity mask contains set bits at positions higher than the total group-local CPU count that was captured during earlier initialization. This fixes customer-reported crashes which have occurred on multi-group machines with heterogenous CPU counts across groups (but the same crash can probably also occur on single-group VMs if CPUs are hot-added and are then manually added to the process affinity mask).
|
Tagging subscribers to this area: @mangod9 Issue DetailsPort of #59129 Hardened ThreadpoolMgr::GateThreadStart against the possibility that the observed group-local affinity mask contains set bits at positions higher than the total group-local CPU count that was captured during earlier initialization. This fixes customer-reported crashes which have occurred on multi-group machines with heterogenous CPU counts across groups (but the same crash can probably also occur on single-group VMs if CPUs are hot-added and are then manually added to the process affinity mask). Customer ImpactWhen the thread pool is used after being idle for a short duration, the system's logical processor count and the process' affinity have increased since the start of the process, and the process is not configured to use all CPU groups, the process may crash. This can potentially happen on multi-CPU-group systems with heterogenous CPU counts across groups where the process may be moved to a different CPU group with more processors, or in systems where CPUs may be hot-added. TestingVerified in the debugger that increases to the affinity mask behave as expected after the change. RiskLow. The change may only increase and not decrease the size of an array, up to a max of 64 elements. I have not seen many of these crashes so far in .NET Core.
|
ghost
locked as resolved and limited conversation to collaborators
2 participants
Port of #59129
Hardened ThreadpoolMgr::GateThreadStart against the possibility that the observed group-local affinity mask contains set bits at positions higher than the total group-local CPU count that was captured during earlier initialization.
This fixes customer-reported crashes which have occurred on multi-group machines with heterogenous CPU counts across groups (but the same crash can probably also occur on single-group VMs if CPUs are hot-added and are then manually added to the process affinity mask).
Customer Impact
When the thread pool is used after being idle for a short duration, the system's logical processor count and the process' affinity have increased since the start of the process, and the process is not configured to use all CPU groups, the process may crash. This can potentially happen on multi-CPU-group systems with heterogenous CPU counts across groups where the process may be moved to a different CPU group with more processors, or in systems where CPUs may be hot-added.
Testing
Verified in the debugger that increases to the affinity mask behave as expected after the change.
Risk
Low. The change may only increase and not decrease the size of an array, up to a max of 64 elements. I have not seen many of these crashes so far in .NET Core.