Test failure: tracing/eventcounter/runtimecounters/runtimecounters.cmd

[v-wenyuxu]

Failed in: runtime-coreclr outerloop 20240724.3

Failed tests:

coreclr osx x64 Checked no_tiered_compilation @ OSX.1200.Amd64.Open
    - tracing/eventcounter/runtimecounters/runtimecounters.cmd

Error message:

 
Return code:      1
Raw output file:      /tmp/helix/working/A3FB08D4/w/B5E80A10/uploads/eventcounter/runtimecounters/output.txt
Raw output:
BEGIN EXECUTION
/tmp/helix/working/A3FB08D4/p/corerun -p System.Reflection.Metadata.MetadataUpdater.IsSupported=false -p System.Runtime.Serialization.EnableUnsafeBinaryFormatterSerialization=true runtimecounters.dll ''
Saw cpu-usage
System.InvalidOperationException: Collection was modified; enumeration operation may not execute.
   at System.Collections.Generic.Dictionary`2.KeyCollection.Enumerator.MoveNext()
   at RuntimeEventCounterTests.RuntimeCounterListener.Verify()
   at RuntimeEventCounterTests.TestRuntimeEventCounter.TestEntryPoint()
   at __GeneratedMainWrapper.Main()
Expected: 100
Actual: 101
END EXECUTION - FAILED
Test failed. Trying to see if dump file was created in /cores since 7/24/2024 12:31:07 PM
Test Harness Exitcode is : 1
To run the test:
Set up CORE_ROOT and run.
> /private/tmp/helix/working/A3FB08D4/w/B5E80A10/e/tracing/tracing/../eventcounter/runtimecounters/runtimecounters.sh

Stack trace:

   at Xunit.Assert.True(Nullable`1 condition, String userMessage) in /_/src/Microsoft.DotNet.XUnitAssert/src/BooleanAsserts.cs:line 146
   at TestLibrary.OutOfProcessTest.RunOutOfProcessTest(String assemblyPath, String testPathPrefix)
   at Program.<<Main>$>g__TestExecutor9|0_10(StreamWriter tempLogSw, StreamWriter statsCsvSw, <>c__DisplayClass0_0&)

Known Issue Error Message

Fill the error message using step by step known issues guidance.

{
  "ErrorMessage": ["InvalidOperationException","RuntimeEventCounterTests"],
  "ErrorPattern": "",
  "BuildRetry": false,
  "ExcludeConsoleLog": false
}

Known issue validation

Build: 🔎 https://dev.azure.com/dnceng-public/public/_build/results?buildId=753860
Error message validated: [InvalidOperationException RuntimeEventCounterTests]
Result validation: ✅ Known issue matched with the provided build.
Validation performed at: 7/25/2024 3:19:50 AM UTC

Report

Build	Definition	Test	Pull Request
755927	dotnet/runtime	tracing/eventcounter/runtimecounters/runtimecounters.cmd	#105513
755583	dotnet/runtime	tracing/eventcounter/runtimecounters/runtimecounters.cmd	#105508
755405	dotnet/runtime	tracing/eventcounter/runtimecounters/runtimecounters.cmd	#103318
754253	dotnet/runtime	tracing/eventcounter/runtimecounters/runtimecounters.cmd	#105341
753860	dotnet/runtime	tracing/eventcounter/runtimecounters/runtimecounters.cmd

Summary

24-Hour Hit Count	7-Day Hit Count	1-Month Count
3	5	5

[jkotas]

This was exposed by #104461. @gbalykov Could you please take a look and submit a fix? The test needs to be updated to use locks or a thread safe collection (ConcurrentDictionary).

[gbalykov]

@jkotas I'm on vacation right now, please revert my PR if fast fix is needed now. I'll take a look after coming back.

[v-wenyuxu]

Failed in: runtime-coreclr r2r 20240724.1

Failed tests:

R2R-CG2 windows arm64 Checked no_tiered_compilation @ Windows.11.Arm64.Open
    - tracing/eventcounter/runtimecounters/runtimecounters.cmd

Error message:

 
Return code:      1
Raw output file:      C:hwB29A09A8wB1D009DDuploadseventcounter
untimecountersoutput.txt
Raw output:
BEGIN EXECUTION
runtimecounters.dll
tracing_common.dll
        2 file(s) copied.
 6:32:49.84
Response file: C:hwB29A09A8wB1D009DDe	racingeventcounter
untimecounters
untimecounters.dll.rsp
C:hwB29A09A8wB1D009DDe	racingeventcounter
untimecountersIL-CG2
untimecounters.dll
-o:C:hwB29A09A8wB1D009DDe	racingeventcounter
untimecounters
untimecounters.dll
--targetarch:arm64
--targetos:windows
--verify-type-and-field-layout
--method-layout:random
-r:C:hwB29A09A8pSystem.*.dll
-r:C:hwB29A09A8pMicrosoft.*.dll
-r:C:hwB29A09A8p\xunit.*.dll
-r:C:hwB29A09A8pmscorlib.dll
-r:C:hwB29A09A8p
etstandard.dll
" "dotnet" "C:hwB29A09A8pcrossgen2crossgen2.dll" @"C:hwB29A09A8wB1D009DDe	racingeventcounter
untimecounters
untimecounters.dll.rsp"   -r:C:hwB29A09A8wB1D009DDe	racing	racingIL-CG2*.dll  -r:C:hwB29A09A8wB1D009DDe	racingeventcounter
untimecountersIL-CG2*.dll"
Emitting R2R PE file: C:hwB29A09A8wB1D009DDe	racingeventcounter
untimecounters
untimecounters.dll
" "dotnet" "C:hwB29A09A8p
2rdump
2rdump.dll" --header --sc --in C:hwB29A09A8wB1D009DDe	racingeventcounter
untimecounters
untimecounters.dll --out C:hwB29A09A8wB1D009DDe	racingeventcounter
untimecounters
untimecounters.dll.r2rdump --val"
 6:32:53.43
 6:32:53.43
Response file: C:hwB29A09A8wB1D009DDe	racingeventcounter
untimecounters	racing_common.dll.rsp
C:hwB29A09A8wB1D009DDe	racingeventcounter
untimecountersIL-CG2	racing_common.dll
-o:C:hwB29A09A8wB1D009DDe	racingeventcounter
untimecounters	racing_common.dll
--targetarch:arm64
--targetos:windows
--verify-type-and-field-layout
--method-layout:random
-r:C:hwB29A09A8pSystem.*.dll
-r:C:hwB29A09A8pMicrosoft.*.dll
-r:C:hwB29A09A8p\xunit.*.dll
-r:C:hwB29A09A8pmscorlib.dll
-r:C:hwB29A09A8p
etstandard.dll
" "dotnet" "C:hwB29A09A8pcrossgen2crossgen2.dll" @"C:hwB29A09A8wB1D009DDe	racingeventcounter
untimecounters	racing_common.dll.rsp"   -r:C:hwB29A09A8wB1D009DDe	racing	racingIL-CG2*.dll  -r:C:hwB29A09A8wB1D009DDe	racingeventcounter
untimecountersIL-CG2*.dll"
Emitting R2R PE file: C:hwB29A09A8wB1D009DDe	racingeventcounter
untimecounters	racing_common.dll
" "dotnet" "C:hwB29A09A8p
2rdump
2rdump.dll" --header --sc --in C:hwB29A09A8wB1D009DDe	racingeventcounter
untimecounters	racing_common.dll --out C:hwB29A09A8wB1D009DDe	racingeventcounter
untimecounters	racing_common.dll.r2rdump --val"
 6:32:57.12
 "C:hwB29A09A8pcorerun.exe" -p "System.Reflection.Metadata.MetadataUpdater.IsSupported=false" -p "System.Runtime.Serialization.EnableUnsafeBinaryFormatterSerialization=true"  runtimecounters.dll 
Saw cpu-usage
System.InvalidOperationException: Collection was modified; enumeration operation may not execute.
   at System.Collections.Generic.Dictionary`2.KeyCollection.Enumerator.MoveNext()
   at RuntimeEventCounterTests.RuntimeCounterListener.Verify()
   at RuntimeEventCounterTests.TestRuntimeEventCounter.TestEntryPoint()
   at __GeneratedMainWrapper.Main()
Expected: 100
Actual: 101
END EXECUTION - FAILED
FAILED
Test failed. Trying to see if dump file was created in C:cores since 7/25/2024 6:32:49 AM
Test Harness Exitcode is : 1
To run the test:
Set up CORE_ROOT and run.
> C:hwB29A09A8wB1D009DDe	racing	racing../eventcounter/runtimecounters/runtimecounters.cmd

Stack trace:

   at Xunit.Assert.True(Nullable`1 condition, String userMessage) in /_/src/Microsoft.DotNet.XUnitAssert/src/BooleanAsserts.cs:line 146
   at Xunit.Assert.True(Boolean condition, String userMessage) in /_/src/Microsoft.DotNet.XUnitAssert/src/BooleanAsserts.cs:line 128
   at TestLibrary.OutOfProcessTest.RunOutOfProcessTest(String assemblyPath, String testPathPrefix)
   at Program.<<Main>$>g__TestExecutor9|0_10(StreamWriter tempLogSw, StreamWriter statsCsvSw, <>c__DisplayClass0_0&)