System.Diagnostics.Tests.ProcessTests.TestCheckChildProcessUserAndGroupIds failing Subset assertion

[ericstj]

Build Information

Build: https://dev.azure.com/dnceng-public/cbb18261-c48f-4abb-8651-8cdcb5474649/_build/results?buildId=425348
Build error leg or test failing: System.Diagnostics.Tests.ProcessTests.TestCheckChildProcessUserAndGroupIds
Pull request: #92730

Error Message

Similar issue reported in past as #28922

Child exception:
  Xunit.Sdk.SubsetException: Assert.Subset() Failure
Expected: HashSet<UInt32> [12, 33, 61, 80, 98, ...]
Actual:   HashSet<UInt32> [80, 12, 20, 61, 701, ...]
   at System.Diagnostics.Tests.ProcessTests.CheckUserAndGroupIds(String userId, String groupId, String groupIdsJoined, String checkGroupsExact) in /_/src/libraries/System.Diagnostics.Process/tests/ProcessTests.Unix.cs:line 571
   at System.Reflection.MethodBaseInvoker.InterpretedInvoke_Method(Object obj, IntPtr* args) in /_/src/mono/System.Private.CoreLib/src/System/Reflection/MethodBaseInvoker.Mono.cs:line 22
   at System.Reflection.MethodBaseInvoker.InvokeDirectByRefWithFewArgs(Object obj, Span`1 copyOfArgs, BindingFlags invokeAttr) in /_/src/libraries/System.Private.CoreLib/src/System/Reflection/MethodBaseInvoker.cs:line 178

Child process:
  System.Diagnostics.Process.Tests, Version=8.0.0.0, Culture=neutral, PublicKeyToken=cc7b13ffcd2ddd51 System.Diagnostics.Tests.ProcessTests Int32 CheckUserAndGroupIds(System.String, System.String, System.String, System.String)

Child arguments:
  504, 80, 12,33,61,80,98,100,204,250,395,398,399,400,701,702,703, False

Fill the error message using step by step known issues guidance.

{
  "ErrorMessage": "",
  "ErrorPattern": "at System.Diagnostics.Tests.ProcessTests.CheckUserAndGroupIds.String userId",
  "BuildRetry": false,
  "ExcludeConsoleLog": true
}

Known issue validation

Build: 🔎 https://dev.azure.com/dnceng-public/public/_build/results?buildId=426977
Error message validated: at System.Diagnostics.Tests.ProcessTests.CheckUserAndGroupIds.String userId
Result validation: ✅ Known issue matched with the provided build.
Validation performed at: 10/4/2023 7:30:00 PM UTC

Report

Summary

24-Hour Hit Count	7-Day Hit Count	1-Month Count
0	0	0

Tagging subscribers to this area: @dotnet/area-system-diagnostics-process
See info in area-owners.md if you want to be subscribed.

Issue Details

---

Build Information

Build: https://dev.azure.com/dnceng-public/cbb18261-c48f-4abb-8651-8cdcb5474649/_build/results?buildId=425348
Build error leg or test failing: System.Diagnostics.Tests.ProcessTests.TestCheckChildProcessUserAndGroupIds
Pull request: #92730

Error Message

Similar issue reported in past as #28922

Child exception:
  Xunit.Sdk.SubsetException: Assert.Subset() Failure
Expected: HashSet<UInt32> [12, 33, 61, 80, 98, ...]
Actual:   HashSet<UInt32> [80, 12, 20, 61, 701, ...]
   at System.Diagnostics.Tests.ProcessTests.CheckUserAndGroupIds(String userId, String groupId, String groupIdsJoined, String checkGroupsExact) in /_/src/libraries/System.Diagnostics.Process/tests/ProcessTests.Unix.cs:line 571
   at System.Reflection.MethodBaseInvoker.InterpretedInvoke_Method(Object obj, IntPtr* args) in /_/src/mono/System.Private.CoreLib/src/System/Reflection/MethodBaseInvoker.Mono.cs:line 22
   at System.Reflection.MethodBaseInvoker.InvokeDirectByRefWithFewArgs(Object obj, Span`1 copyOfArgs, BindingFlags invokeAttr) in /_/src/libraries/System.Private.CoreLib/src/System/Reflection/MethodBaseInvoker.cs:line 178

Child process:
  System.Diagnostics.Process.Tests, Version=8.0.0.0, Culture=neutral, PublicKeyToken=cc7b13ffcd2ddd51 System.Diagnostics.Tests.ProcessTests Int32 CheckUserAndGroupIds(System.String, System.String, System.String, System.String)

Child arguments:
  504, 80, 12,33,61,80,98,100,204,250,395,398,399,400,701,702,703, False

Fill the error message using step by step known issues guidance.

{
  "ErrorMessage": "at System.Diagnostics.Tests.ProcessTests.CheckUserAndGroupIds(String userId, String groupId, String groupIdsJoined, String checkGroupsExact)",
  "ErrorPattern": "",
  "BuildRetry": false,
  "ExcludeConsoleLog": true
}

Author:	ericstj
Assignees:	-
Labels:	area-System.Diagnostics.Process, blocking-clean-ci, untriaged, Known Build Error
Milestone:	-

[carlossanlop]

Hit this in 8.0. Not sure why KnownBuildError isn't linking the PRs:

• 8.0 PR: [release/8.0] Fix deadlock in EventPipeEventDispatcher #92912
• Queue: Libraries Test Run release coreclr osx x64 Debug
• Job: https://dev.azure.com/dnceng-public/public/_build/results?buildId=425151&view=logs&j=5da4f169-faec-5863-fdf3-ac008545b5e6&t=04587263-8799-5ed0-c746-7c81bb983f01&l=54
• Log: https://helixre107v0xdeko0k025g8.blob.core.windows.net/dotnet-runtime-refs-pull-92912-merge-ed5754bfc9b0404897/System.Diagnostics.Process.Tests/1/console.24960f7a.log?helixlogtype=result
• Output:

Expand

===========================================================================================================
/private/tmp/helix/working/B1790977/w/A4B20994/e /private/tmp/helix/working/B1790977/w/A4B20994/e
  Discovering: System.Diagnostics.Process.Tests (method display = ClassAndMethod, method display options = None)
  Discovered:  System.Diagnostics.Process.Tests (found 250 of 322 test cases)
  Starting:    System.Diagnostics.Process.Tests (parallel test collections = on, max threads = 12)
    System.Diagnostics.Tests.ProcessStartInfoTests.ShellExecute_Nano_Fails_Start [SKIP]
      Condition(s) not met: "IsWindowsNanoServer"
Darwin
    System.Diagnostics.Tests.ProcessTests.TestBasePriorityOnUnix [SKIP]
      Condition(s) not met: "IsPrivilegedProcess"
Unhandled exception. Xunit.Sdk.SubsetException: Assert.Subset() Failure
Expected: HashSet<UInt32> [12, 33, 61, 80, 98, ...]
Actual:   HashSet<UInt32> [80, 12, 20, 61, 703, ...]
   at Xunit.Assert.Subset[T](ISet`1 expectedSuperset, ISet`1 actual) in /_/src/xunit.assert/Asserts/SetAsserts.cs:line 71
   at System.Diagnostics.Tests.ProcessTests.CheckUserAndGroupIds(String userId, String groupId, String groupIdsJoined, String checkGroupsExact) in /_/src/libraries/System.Diagnostics.Process/tests/ProcessTests.Unix.cs:line 571
   at System.RuntimeMethodHandle.InvokeMethod(Object target, Void** arguments, Signature sig, Boolean isConstructor)
   at System.Reflection.MethodBaseInvoker.InterpretedInvoke_Method(Object obj, IntPtr* args) in /_/src/coreclr/System.Private.CoreLib/src/System/Reflection/MethodBaseInvoker.CoreCLR.cs:line 36
   at System.Reflection.MethodBaseInvoker.InvokeDirectByRefWithFewArgs(Object obj, Span`1 copyOfArgs, BindingFlags invokeAttr) in /_/src/libraries/System.Private.CoreLib/src/System/Reflection/MethodBaseInvoker.cs:line 178
--- End of stack trace from previous location ---
   at Microsoft.DotNet.RemoteExecutor.Program.Main(String[] args) in /_/src/Microsoft.DotNet.RemoteExecutor/src/Program.cs:line 97
    System.Diagnostics.Tests.ProcessTests.TestCheckChildProcessUserAndGroupIds [FAIL]
      Microsoft.DotNet.RemoteExecutor.RemoteExecutionException : Remote process failed with an unhandled exception.
      Stack Trace:
        
        Child exception:
          Xunit.Sdk.SubsetException: Assert.Subset() Failure
        Expected: HashSet<UInt32> [12, 33, 61, 80, 98, ...]
        Actual:   HashSet<UInt32> [80, 12, 20, 61, 703, ...]
        /_/src/libraries/System.Diagnostics.Process/tests/ProcessTests.Unix.cs(571,0): at System.Diagnostics.Tests.ProcessTests.CheckUserAndGroupIds(String userId, String groupId, String groupIdsJoined, String checkGroupsExact)
           at System.RuntimeMethodHandle.InvokeMethod(Object target, Void** arguments, Signature sig, Boolean isConstructor)
        /_/src/coreclr/System.Private.CoreLib/src/System/Reflection/MethodBaseInvoker.CoreCLR.cs(36,0): at System.Reflection.MethodBaseInvoker.InterpretedInvoke_Method(Object obj, IntPtr* args)
        /_/src/libraries/System.Private.CoreLib/src/System/Reflection/MethodBaseInvoker.cs(178,0): at System.Reflection.MethodBaseInvoker.InvokeDirectByRefWithFewArgs(Object obj, Span`1 copyOfArgs, BindingFlags invokeAttr)
        
        Child process:
          System.Diagnostics.Process.Tests, Version=8.0.0.0, Culture=neutral, PublicKeyToken=cc7b13ffcd2ddd51 System.Diagnostics.Tests.ProcessTests Int32 CheckUserAndGroupIds(System.String, System.String, System.String, System.String)
        
        Child arguments:
          504, 80, 12,33,61,80,98,100,204,250,395,398,399,400,701,702,703, False
        
    System.Diagnostics.Tests.ProcessTests.TestPriorityClassUnix [SKIP]
      Condition(s) not met: "IsPrivilegedProcess"
    System.Diagnostics.Tests.ProcessTests.TestProcessRecycledPid [SKIP]
      Condition(s) not met: "IsStressModeEnabledAndRemoteExecutorSupported"
    System.Diagnostics.Tests.ProcessTests.TestCheckChildProcessUserAndGroupIdsElevated [SKIP]
      Condition(s) not met: "IsRemoteExecutorSupportedAndPrivilegedProcess"
  Finished:    System.Diagnostics.Process.Tests
=== TEST EXECUTION SUMMARY ===
   System.Diagnostics.Process.Tests  Total: 306, Errors: 0, Failed: 1, Skipped: 5, Time: 24.023s
/private/tmp/helix/working/B1790977/w/A4B20994/e
----- end Mon Oct 2 19:19:37 EDT 2023 ----- exit code 1 ----------------------------------------------------------

[buyaa-n]

Same failure found in #92846, Build Analysis might link this issue automatically if we remove the ProcessTests.Unix.cs(571,0): part from the "ErrorMessage" as that part is not visible in the Build Analysis tab:

UPDATE: looks it was originally: "ErrorMessage": "at System.Diagnostics.Tests.ProcessTests.CheckUserAndGroupIds(String userId, String groupId, String groupIdsJoined, String checkGroupsExact)", and apparently the automation still did not work

[carlossanlop]

Hit in 7.0:

• PR: [release/7.0] Update dependencies from dotnet/emsdk #92992
• Build: https://dev.azure.com/dnceng-public/public/_build/results?buildId=426977
• Queue: Libraries Test Run release mono OSX x64 Debug
• Run: https://dev.azure.com/dnceng-public/public/_build/results?buildId=426977&view=logs&j=c6f8dc49-92a1-5760-c098-ba97b8142bfb&t=22b0078b-0469-5ba6-8725-2121fdbae049&l=55
• Log: https://helixre107v0xdeko0k025g8.blob.core.windows.net/dotnet-runtime-refs-pull-92992-merge-745467eb271f41b1b8/System.Diagnostics.Process.Tests/1/console.96a3bfb6.log?helixlogtype=result
• Output:

Unhandled Exception:
Xunit.Sdk.SubsetException: Assert.Subset() Failure
Expected: HashSet<UInt32> [12, 33, 61, 80, 98, ...]
Actual:   HashSet<UInt32> [80, 12, 20, 61, 701, ...]
   at Xunit.Assert.Subset[UInt32](ISet`1 expectedSuperset, ISet`1 actual) in /_/src/xunit.assert/Asserts/SetAsserts.cs:line 72
   at System.Diagnostics.Tests.ProcessTests.CheckUserAndGroupIds(String userId, String groupId, String groupIdsJoined, String checkGroupsExact) in /_/src/libraries/System.Diagnostics.Process/tests/ProcessTests.Unix.cs:line 568
   at Microsoft.DotNet.RemoteExecutor.Program.Main(String[] args) in /_/src/Microsoft.DotNet.RemoteExecutor/src/Program.cs:line 97
[ERROR] FATAL UNHANDLED EXCEPTION: Xunit.Sdk.SubsetException: Assert.Subset() Failure
Expected: HashSet<UInt32> [12, 33, 61, 80, 98, ...]
Actual:   HashSet<UInt32> [80, 12, 20, 61, 701, ...]
   at Xunit.Assert.Subset[UInt32](ISet`1 expectedSuperset, ISet`1 actual) in /_/src/xunit.assert/Asserts/SetAsserts.cs:line 72
   at System.Diagnostics.Tests.ProcessTests.CheckUserAndGroupIds(String userId, String groupId, String groupIdsJoined, String checkGroupsExact) in /_/src/libraries/System.Diagnostics.Process/tests/ProcessTests.Unix.cs:line 568
   at Microsoft.DotNet.RemoteExecutor.Program.Main(String[] args) in /_/src/Microsoft.DotNet.RemoteExecutor/src/Program.cs:line 97
    System.Diagnostics.Tests.ProcessTests.TestCheckChildProcessUserAndGroupIds [FAIL]
      Microsoft.DotNet.RemoteExecutor.RemoteExecutionException : Remote process failed with an unhandled exception.
      Stack Trace:
        
        Child exception:
          Xunit.Sdk.SubsetException: Assert.Subset() Failure
        Expected: HashSet<UInt32> [12, 33, 61, 80, 98, ...]
        Actual:   HashSet<UInt32> [80, 12, 20, 61, 701, ...]
        /_/src/libraries/System.Diagnostics.Process/tests/ProcessTests.Unix.cs(568,0): at System.Diagnostics.Tests.ProcessTests.CheckUserAndGroupIds(String userId, String groupId, String groupIdsJoined, String checkGroupsExact)
        /_/src/mono/System.Private.CoreLib/src/System/Reflection/MethodInvoker.Mono.cs(33,0): at System.Reflection.MethodInvoker.InterpretedInvoke(Object obj, Span`1 args, BindingFlags invokeAttr)
        
        Child process:
          System.Diagnostics.Process.Tests, Version=7.0.0.0, Culture=neutral, PublicKeyToken=cc7b13ffcd2ddd51 System.Diagnostics.Tests.ProcessTests Int32 CheckUserAndGroupIds(System.String, System.String, System.String, System.String)
        
        Child arguments:
          504, 80, 12,33,61,80,98,100,204,250,395,398,399,400,701,702,703, False