[wasm] Wasm.Build.Tests failing on windows

[radekdoulik]

These tests fail on helix/windows and also locally, in different way.

On helix:

C:\helix\work\workitem>set WasmBuildSupportDir=C:\helix\work\correlation\build   & mkdir C:\helix\work\correlation\build   & set DOTNET_SYSTEM_GLOBALIZATION_INVARIANT=1   & set TEST_LOG_PATH="C:\helix\work\workitem\uploads\xharness-output"\logs   & set HARNESS_RUNNER=dotnet.exe exec "C:\helix\work\correlation\microsoft.dotnet.xharness.cli\1.0.0-prerelease.21263.1\tools\net6.0\any\Microsoft.DotNet.XHarness.CLI.dll"   & dotnet.exe exec xunit.console.dll Wasm.Build.Tests.dll -xml "C:\helix\work\workitem\uploads\xharness-output"\testResults.xml -nocolor -verbose -notrait category=IgnoreForCI -notrait category=OuterLoop -notrait category=failing  
Access is denied.
Microsoft.DotNet.XUnitConsoleRunner v2.5.0 (64-bit .NET 6.0.0-preview.3.21201.4)
  Discovering: Wasm.Build.Tests (method display = ClassAndMethod, method display options = None)
  Discovered:  Wasm.Build.Tests (found 14 test cases)
  Starting:    Wasm.Build.Tests (parallel test collections = on, max threads = 2)
    Wasm.Build.Tests.RebuildTests.NoOpRebuild [STARTING]
    Wasm.Build.Tests.MainWithArgsTests.TopLevelWithArgs [STARTING]
============== DELETING THE BUILDS =============
      System.TypeInitializationException : The type initializer for 'Wasm.Build.Tests.BuildTestBase' threw an exception.
      ---- System.Exception : Could not find the solution root, or a build dir: C:\helix\work\correlation\build 
      Stack Trace:
    Wasm.Build.Tests.RebuildTests.NoOpRebuild [FAIL]
        /_/src/tests/BuildWasmApps/Wasm.Build.Tests/BuildTestBase.cs(123,0): at Wasm.Build.Tests.BuildTestBase.ConfigWithAOTData(Boolean aot)
        /_/src/tests/BuildWasmApps/Wasm.Build.Tests/MainWithArgsTests.cs(22,0): at Wasm.Build.Tests.MainWithArgsTests.MainWithArgsTestData(Boolean aot, RunHost host)
        ----- Inner Stack Trace -----
        /_/src/tests/BuildWasmApps/Wasm.Build.Tests/BuildTestBase.cs(57,0): at Wasm.Build.Tests.BuildTestBase..cctor()
    Wasm.Build.Tests.RebuildTests.NoOpRebuild [FINISHED] Time: 0s
...
=== TEST EXECUTION SUMMARY ===
   Wasm.Build.Tests  Total: 14, Errors: 0, Failed: 14, Skipped: 0, Time: 0.107s

Local:

  Microsoft.DotNet.XUnitConsoleRunner v2.5.0 (64-bit .NET Core 3.1.13)
    Discovering: Wasm.Build.Tests (method display = ClassAndMethod, method display options = None)
     Wasm.Build.Tests: Exception during discovery:
  System.TypeLoadException: Could not load type 'System.Diagnostics.CodeAnalysis.MemberNotNullAttribute' from assembly 'System.Runtime, Version=6.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a'.
     at System.ModuleHandle.ResolveMethodHandleInternalCore(RuntimeModule module, Int32 methodToken, IntPtr[] typeInstantiationContext, Int32 typeInstCount, IntPtr[] methodInstantiationContext, Int32 methodInstCount)
     at System.ModuleHandle.ResolveMethodHandleInternal(RuntimeModule module, Int32 methodToken, RuntimeTypeHandle[] typeInstantiationContext, RuntimeTypeHandle[] methodInstantiationContext)
     at System.Reflection.CustomAttributeData..ctor(RuntimeModule scope, MetadataToken caCtorToken, ConstArray& blob)
     at System.Reflection.CustomAttributeData.GetCustomAttributes(RuntimeModule module, Int32 tkTarget)
     at System.Reflection.CustomAttributeData.GetCustomAttributesInternal(RuntimeMethodInfo target)
     at System.Reflection.RuntimeMethodInfo.GetCustomAttributesData()
     at System.Reflection.MemberInfo.get_CustomAttributes()
     at Xunit.Sdk.ReflectionMethodInfo.GetCustomAttributes(MethodInfo method, Type attributeType, AttributeUsageAttribute attributeUsage) in C:\Dev\xunit\xunit\src\xunit.execution\Sdk\Reflection\ReflectionMethodInfo.cs:line 93
     at Xunit.Sdk.ReflectionMethodInfo.GetCustomAttributes(MethodInfo method, String assemblyQualifiedAttributeTypeName) in C:\Dev\xunit\xunit\src\xunit.execution\Sdk\Reflection\ReflectionMethodInfo.cs:line 87
     at Xunit.Sdk.ReflectionMethodInfo.GetCustomAttributes(String assemblyQualifiedAttributeTypeName) in C:\Dev\xunit\xunit\src\xunit.execution\Sdk\Reflection\ReflectionMethodInfo.cs:line 80
     at Xunit.Sdk.XunitTestFrameworkDiscoverer.FindTestsForMethod(ITestMethod testMethod, Boolean includeSourceInformation, IMessageBus messageBus, ITestFrameworkDiscoveryOptions discoveryOptions) in C:\Dev\xunit\xunit\src\xunit.execution\Sdk\Frameworks\XunitTestFrameworkDiscoverer.cs:line 84
     at Xunit.Sdk.XunitTestFrameworkDiscoverer.FindTestsForType(ITestClass testClass, Boolean includeSourceInformation, IMessageBus messageBus, ITestFrameworkDiscoveryOptions discoveryOptions) in C:\Dev\xunit\xunit\src\xunit.execution\Sdk\Frameworks\XunitTestFrameworkDiscoverer.cs:line 132
     at Xunit.Sdk.TestFrameworkDiscoverer.FindTestsForTypeAndWrapExceptions(ITestClass testClass, Boolean includeSourceInformation, IMessageBus messageBus, ITestFrameworkDiscoveryOptions discoveryOptions) in C:\Dev\xunit\xunit\src\xunit.execution\Sdk\Frameworks\TestFrameworkDiscoverer.cs:line 156
     Wasm.Build.Tests: Exception during discovery:
  System.TypeLoadException: Could not load type 'System.Diagnostics.CodeAnalysis.MemberNotNullAttribute' from assembly 'System.Runtime, Version=6.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a'.
     at System.ModuleHandle.ResolveMethodHandleInternalCore(RuntimeModule module, Int32 methodToken, IntPtr[] typeInstantiationContext, Int32 typeInstCount, IntPtr[] methodInstantiationContext, Int32 methodInstCount)
     at System.ModuleHandle.ResolveMethodHandleInternal(RuntimeModule module, Int32 methodToken, RuntimeTypeHandle[] typeInstantiationContext, RuntimeTypeHandle[] methodInstantiationContext)
     at System.Reflection.CustomAttributeData..ctor(RuntimeModule scope, MetadataToken caCtorToken, ConstArray& blob)
     at System.Reflection.CustomAttributeData.GetCustomAttributes(RuntimeModule module, Int32 tkTarget)
     at System.Reflection.CustomAttributeData.GetCustomAttributesInternal(RuntimeMethodInfo target)
     at System.Reflection.RuntimeMethodInfo.GetCustomAttributesData()
     at System.Reflection.MemberInfo.get_CustomAttributes()
     at Xunit.Sdk.ReflectionMethodInfo.GetCustomAttributes(MethodInfo method, Type attributeType, AttributeUsageAttribute attributeUsage) in C:\Dev\xunit\xunit\src\xunit.execution\Sdk\Reflection\ReflectionMethodInfo.cs:line 93
     at Xunit.Sdk.ReflectionMethodInfo.GetCustomAttributes(MethodInfo method, String assemblyQualifiedAttributeTypeName) in C:\Dev\xunit\xunit\src\xunit.execution\Sdk\Reflection\ReflectionMethodInfo.cs:line 87
     at Xunit.Sdk.ReflectionMethodInfo.GetCustomAttributes(String assemblyQualifiedAttributeTypeName) in C:\Dev\xunit\xunit\src\xunit.execution\Sdk\Reflection\ReflectionMethodInfo.cs:line 80
     at Xunit.Sdk.XunitTestFrameworkDiscoverer.FindTestsForMethod(ITestMethod testMethod, Boolean includeSourceInformation, IMessageBus messageBus, ITestFrameworkDiscoveryOptions discoveryOptions) in C:\Dev\xunit\xunit\src\xunit.execution\Sdk\Frameworks\XunitTestFrameworkDiscoverer.cs:line 84
     at Xunit.Sdk.XunitTestFrameworkDiscoverer.FindTestsForType(ITestClass testClass, Boolean includeSourceInformation, IMessageBus messageBus, ITestFrameworkDiscoveryOptions discoveryOptions) in C:\Dev\xunit\xunit\src\xunit.execution\Sdk\Frameworks\XunitTestFrameworkDiscoverer.cs:line 132
     at Xunit.Sdk.TestFrameworkDiscoverer.FindTestsForTypeAndWrapExceptions(ITestClass testClass, Boolean includeSourceInformation, IMessageBus messageBus, ITestFrameworkDiscoveryOptions discoveryOptions) in C:\Dev\xunit\xunit\src\xunit.execution\Sdk\Frameworks\TestFrameworkDiscoverer.cs:line 156
     Wasm.Build.Tests: Exception during discovery:
  System.TypeLoadException: Could not load type 'System.Diagnostics.CodeAnalysis.MemberNotNullAttribute' from assembly 'System.Runtime, Version=6.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a'.
     at System.ModuleHandle.ResolveMethodHandleInternalCore(RuntimeModule module, Int32 methodToken, IntPtr[] typeInstantiationContext, Int32 typeInstCount, IntPtr[] methodInstantiationContext, Int32 methodInstCount)
     at System.ModuleHandle.ResolveMethodHandleInternal(RuntimeModule module, Int32 methodToken, RuntimeTypeHandle[] typeInstantiationContext, RuntimeTypeHandle[] methodInstantiationContext)
     at System.Reflection.CustomAttributeData..ctor(RuntimeModule scope, MetadataToken caCtorToken, ConstArray& blob)
     at System.Reflection.CustomAttributeData.GetCustomAttributes(RuntimeModule module, Int32 tkTarget)
     at System.Reflection.CustomAttributeData.GetCustomAttributesInternal(RuntimeMethodInfo target)
     at System.Reflection.RuntimeMethodInfo.GetCustomAttributesData()
     at System.Reflection.MemberInfo.get_CustomAttributes()
     at Xunit.Sdk.ReflectionMethodInfo.GetCustomAttributes(MethodInfo method, Type attributeType, AttributeUsageAttribute attributeUsage) in C:\Dev\xunit\xunit\src\xunit.execution\Sdk\Reflection\ReflectionMethodInfo.cs:line 93
     at Xunit.Sdk.ReflectionMethodInfo.GetCustomAttributes(MethodInfo method, String assemblyQualifiedAttributeTypeName) in C:\Dev\xunit\xunit\src\xunit.execution\Sdk\Reflection\ReflectionMethodInfo.cs:line 87
     at Xunit.Sdk.ReflectionMethodInfo.GetCustomAttributes(String assemblyQualifiedAttributeTypeName) in C:\Dev\xunit\xunit\src\xunit.execution\Sdk\Reflection\ReflectionMethodInfo.cs:line 80
     at Xunit.Sdk.XunitTestFrameworkDiscoverer.FindTestsForMethod(ITestMethod testMethod, Boolean includeSourceInformation, IMessageBus messageBus, ITestFrameworkDiscoveryOptions discoveryOptions) in C:\Dev\xunit\xunit\src\xunit.execution\Sdk\Frameworks\XunitTestFrameworkDiscoverer.cs:line 84
     at Xunit.Sdk.XunitTestFrameworkDiscoverer.FindTestsForType(ITestClass testClass, Boolean includeSourceInformation, IMessageBus messageBus, ITestFrameworkDiscoveryOptions discoveryOptions) in C:\Dev\xunit\xunit\src\xunit.execution\Sdk\Frameworks\XunitTestFrameworkDiscoverer.cs:line 132
     at Xunit.Sdk.TestFrameworkDiscoverer.FindTestsForTypeAndWrapExceptions(ITestClass testClass, Boolean includeSourceInformation, IMessageBus messageBus, ITestFrameworkDiscoveryOptions discoveryOptions) in C:\Dev\xunit\xunit\src\xunit.execution\Sdk\Frameworks\TestFrameworkDiscoverer.cs:line 156
     Wasm.Build.Tests: Exception during discovery:
  System.TypeLoadException: Could not load type 'System.Diagnostics.CodeAnalysis.MemberNotNullAttribute' from assembly 'System.Runtime, Version=6.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a'.
     at System.ModuleHandle.ResolveMethodHandleInternalCore(RuntimeModule module, Int32 methodToken, IntPtr[] typeInstantiationContext, Int32 typeInstCount, IntPtr[] methodInstantiationContext, Int32 methodInstCount)
     at System.ModuleHandle.ResolveMethodHandleInternal(RuntimeModule module, Int32 methodToken, RuntimeTypeHandle[] typeInstantiationContext, RuntimeTypeHandle[] methodInstantiationContext)
     at System.Reflection.CustomAttributeData..ctor(RuntimeModule scope, MetadataToken caCtorToken, ConstArray& blob)
     at System.Reflection.CustomAttributeData.GetCustomAttributes(RuntimeModule module, Int32 tkTarget)
     at System.Reflection.CustomAttributeData.GetCustomAttributesInternal(RuntimeMethodInfo target)
     at System.Reflection.RuntimeMethodInfo.GetCustomAttributesData()
     at System.Reflection.MemberInfo.get_CustomAttributes()
     at Xunit.Sdk.ReflectionMethodInfo.GetCustomAttributes(MethodInfo method, Type attributeType, AttributeUsageAttribute attributeUsage) in C:\Dev\xunit\xunit\src\xunit.execution\Sdk\Reflection\ReflectionMethodInfo.cs:line 93
     at Xunit.Sdk.ReflectionMethodInfo.GetCustomAttributes(MethodInfo method, String assemblyQualifiedAttributeTypeName) in C:\Dev\xunit\xunit\src\xunit.execution\Sdk\Reflection\ReflectionMethodInfo.cs:line 87
     at Xunit.Sdk.ReflectionMethodInfo.GetCustomAttributes(String assemblyQualifiedAttributeTypeName) in C:\Dev\xunit\xunit\src\xunit.execution\Sdk\Reflection\ReflectionMethodInfo.cs:line 80
     at Xunit.Sdk.XunitTestFrameworkDiscoverer.FindTestsForMethod(ITestMethod testMethod, Boolean includeSourceInformation, IMessageBus messageBus, ITestFrameworkDiscoveryOptions discoveryOptions) in C:\Dev\xunit\xunit\src\xunit.execution\Sdk\Frameworks\XunitTestFrameworkDiscoverer.cs:line 84
     at Xunit.Sdk.XunitTestFrameworkDiscoverer.FindTestsForType(ITestClass testClass, Boolean includeSourceInformation, IMessageBus messageBus, ITestFrameworkDiscoveryOptions discoveryOptions) in C:\Dev\xunit\xunit\src\xunit.execution\Sdk\Frameworks\XunitTestFrameworkDiscoverer.cs:line 132
     at Xunit.Sdk.TestFrameworkDiscoverer.FindTestsForTypeAndWrapExceptions(ITestClass testClass, Boolean includeSourceInformation, IMessageBus messageBus, ITestFrameworkDiscoveryOptions discoveryOptions) in C:\Dev\xunit\xunit\src\xunit.execution\Sdk\Frameworks\TestFrameworkDiscoverer.cs:line 156
     Wasm.Build.Tests: Exception during discovery:
  System.TypeLoadException: Could not load type 'System.Diagnostics.CodeAnalysis.MemberNotNullAttribute' from assembly 'System.Runtime, Version=6.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a'.
     at System.ModuleHandle.ResolveMethodHandleInternalCore(RuntimeModule module, Int32 methodToken, IntPtr[] typeInstantiationContext, Int32 typeInstCount, IntPtr[] methodInstantiationContext, Int32 methodInstCount)
     at System.ModuleHandle.ResolveMethodHandleInternal(RuntimeModule module, Int32 methodToken, RuntimeTypeHandle[] typeInstantiationContext, RuntimeTypeHandle[] methodInstantiationContext)
     at System.Reflection.CustomAttributeData..ctor(RuntimeModule scope, MetadataToken caCtorToken, ConstArray& blob)
     at System.Reflection.CustomAttributeData.GetCustomAttributes(RuntimeModule module, Int32 tkTarget)
     at System.Reflection.CustomAttributeData.GetCustomAttributesInternal(RuntimeMethodInfo target)
     at System.Reflection.RuntimeMethodInfo.GetCustomAttributesData()
     at System.Reflection.MemberInfo.get_CustomAttributes()
     at Xunit.Sdk.ReflectionMethodInfo.GetCustomAttributes(MethodInfo method, Type attributeType, AttributeUsageAttribute attributeUsage) in C:\Dev\xunit\xunit\src\xunit.execution\Sdk\Reflection\ReflectionMethodInfo.cs:line 93
     at Xunit.Sdk.ReflectionMethodInfo.GetCustomAttributes(MethodInfo method, String assemblyQualifiedAttributeTypeName) in C:\Dev\xunit\xunit\src\xunit.execution\Sdk\Reflection\ReflectionMethodInfo.cs:line 87
     at Xunit.Sdk.ReflectionMethodInfo.GetCustomAttributes(String assemblyQualifiedAttributeTypeName) in C:\Dev\xunit\xunit\src\xunit.execution\Sdk\Reflection\ReflectionMethodInfo.cs:line 80
     at Xunit.Sdk.XunitTestFrameworkDiscoverer.FindTestsForMethod(ITestMethod testMethod, Boolean includeSourceInformation, IMessageBus messageBus, ITestFrameworkDiscoveryOptions discoveryOptions) in C:\Dev\xunit\xunit\src\xunit.execution\Sdk\Frameworks\XunitTestFrameworkDiscoverer.cs:line 84
     at Xunit.Sdk.XunitTestFrameworkDiscoverer.FindTestsForType(ITestClass testClass, Boolean includeSourceInformation, IMessageBus messageBus, ITestFrameworkDiscoveryOptions discoveryOptions) in C:\Dev\xunit\xunit\src\xunit.execution\Sdk\Frameworks\XunitTestFrameworkDiscoverer.cs:line 132
     at Xunit.Sdk.TestFrameworkDiscoverer.FindTestsForTypeAndWrapExceptions(ITestClass testClass, Boolean includeSourceInformation, IMessageBus messageBus, ITestFrameworkDiscoveryOptions discoveryOptions) in C:\Dev\xunit\xunit\src\xunit.execution\Sdk\Frameworks\TestFrameworkDiscoverer.cs:line 156
    Discovered:  Wasm.Build.Tests (found 14 test cases)
    Starting:    Wasm.Build.Tests (parallel test collections = on, max threads = 48)
      Wasm.Build.Tests.RebuildTests.NoOpRebuild [FAIL]
        System.TypeInitializationException : The type initializer for 'Wasm.Build.Tests.BuildTestBase' threw an exception.
        ---- System.DllNotFoundException : Unable to load DLL 'libSystem.Native' or one of its dependencies: The specified module could not be found. (0x8007007E)
        Stack Trace:
          C:\Users\rodo\git\wa-main\src\tests\BuildWasmApps\Wasm.Build.Tests\BuildTestBase.cs(123,0): at Wasm.Build.Tests.BuildTestBase.ConfigWithAOTData(Boolean aot)
          C:\Users\rodo\git\wa-main\src\tests\BuildWasmApps\Wasm.Build.Tests\InvariantGlobalizationTests.cs(21,0): at Wasm.Build.Tests.InvariantGlobalizationTests.InvariantGlobalizationTestData(Boolean aot, RunHost host)
          ----- Inner Stack Trace -----
             at Interop.Sys.Stat(Byte& path, FileStatus& output)
             at Interop.Sys.Stat(ReadOnlySpan`1 path, FileStatus& output)
             at System.IO.FileSystem.FileExists(ReadOnlySpan`1 fullPath, Int32 fileType, ErrorInfo& errorInfo)
             at System.IO.FileSystem.FileExists(ReadOnlySpan`1 fullPath)
             at System.IO.File.Exists(String path)
          C:\Users\rodo\git\wa-main\src\tests\BuildWasmApps\Wasm.Build.Tests\BuildTestBase.cs(44,0): at Wasm.Build.Tests.BuildTestBase..cctor()
      Wasm.Build.Tests.NativeBuildTests.Relinking_ErrorWhenMissingEMSDK [FAIL]
        System.TypeInitializationException : The type initializer for 'Wasm.Build.Tests.BuildTestBase' threw an exception.
        ---- System.DllNotFoundException : Unable to load DLL 'libSystem.Native' or one of its dependencies: The specified module could not be found. (0x8007007E)
        Stack Trace:
          C:\Users\rodo\git\wa-main\src\tests\BuildWasmApps\Wasm.Build.Tests\BuildTestBase.cs(123,0): at Wasm.Build.Tests.BuildTestBase.ConfigWithAOTData(Boolean aot)
          C:\Users\rodo\git\wa-main\src\tests\BuildWasmApps\Wasm.Build.Tests\InvariantGlobalizationTests.cs(21,0): at Wasm.Build.Tests.InvariantGlobalizationTests.InvariantGlobalizationTestData(Boolean aot, RunHost host)
          ----- Inner Stack Trace -----
             at Interop.Sys.Stat(Byte& path, FileStatus& output)
             at Interop.Sys.Stat(ReadOnlySpan`1 path, FileStatus& output)
             at System.IO.FileSystem.FileExists(ReadOnlySpan`1 fullPath, Int32 fileType, ErrorInfo& errorInfo)
             at System.IO.FileSystem.FileExists(ReadOnlySpan`1 fullPath)
             at System.IO.File.Exists(String path)
          C:\Users\rodo\git\wa-main\src\tests\BuildWasmApps\Wasm.Build.Tests\BuildTestBase.cs(44,0): at Wasm.Build.Tests.BuildTestBase..cctor()
      Wasm.Build.Tests.MainWithArgsTests.TopLevelWithArgs [FAIL]
...
  === TEST EXECUTION SUMMARY ===
     Wasm.Build.Tests  Total: 14, Errors: 5, Failed: 14, Skipped: 0, Time: 0.094s

Tagging subscribers to 'arch-wasm': @lewing
See info in area-owners.md if you want to be subscribed.

Issue Details

---

These tests fail on helix/windows and also locally, in different way.

On helix:

C:\helix\work\workitem>set WasmBuildSupportDir=C:\helix\work\correlation\build   & mkdir C:\helix\work\correlation\build   & set DOTNET_SYSTEM_GLOBALIZATION_INVARIANT=1   & set TEST_LOG_PATH="C:\helix\work\workitem\uploads\xharness-output"\logs   & set HARNESS_RUNNER=dotnet.exe exec "C:\helix\work\correlation\microsoft.dotnet.xharness.cli\1.0.0-prerelease.21263.1\tools\net6.0\any\Microsoft.DotNet.XHarness.CLI.dll"   & dotnet.exe exec xunit.console.dll Wasm.Build.Tests.dll -xml "C:\helix\work\workitem\uploads\xharness-output"\testResults.xml -nocolor -verbose -notrait category=IgnoreForCI -notrait category=OuterLoop -notrait category=failing  
Access is denied.
Microsoft.DotNet.XUnitConsoleRunner v2.5.0 (64-bit .NET 6.0.0-preview.3.21201.4)
  Discovering: Wasm.Build.Tests (method display = ClassAndMethod, method display options = None)
  Discovered:  Wasm.Build.Tests (found 14 test cases)
  Starting:    Wasm.Build.Tests (parallel test collections = on, max threads = 2)
    Wasm.Build.Tests.RebuildTests.NoOpRebuild [STARTING]
    Wasm.Build.Tests.MainWithArgsTests.TopLevelWithArgs [STARTING]
============== DELETING THE BUILDS =============
      System.TypeInitializationException : The type initializer for 'Wasm.Build.Tests.BuildTestBase' threw an exception.
      ---- System.Exception : Could not find the solution root, or a build dir: C:\helix\work\correlation\build 
      Stack Trace:
    Wasm.Build.Tests.RebuildTests.NoOpRebuild [FAIL]
        /_/src/tests/BuildWasmApps/Wasm.Build.Tests/BuildTestBase.cs(123,0): at Wasm.Build.Tests.BuildTestBase.ConfigWithAOTData(Boolean aot)
        /_/src/tests/BuildWasmApps/Wasm.Build.Tests/MainWithArgsTests.cs(22,0): at Wasm.Build.Tests.MainWithArgsTests.MainWithArgsTestData(Boolean aot, RunHost host)
        ----- Inner Stack Trace -----
        /_/src/tests/BuildWasmApps/Wasm.Build.Tests/BuildTestBase.cs(57,0): at Wasm.Build.Tests.BuildTestBase..cctor()
    Wasm.Build.Tests.RebuildTests.NoOpRebuild [FINISHED] Time: 0s
...
=== TEST EXECUTION SUMMARY ===
   Wasm.Build.Tests  Total: 14, Errors: 0, Failed: 14, Skipped: 0, Time: 0.107s

Local:

  Microsoft.DotNet.XUnitConsoleRunner v2.5.0 (64-bit .NET Core 3.1.13)
    Discovering: Wasm.Build.Tests (method display = ClassAndMethod, method display options = None)
     Wasm.Build.Tests: Exception during discovery:
  System.TypeLoadException: Could not load type 'System.Diagnostics.CodeAnalysis.MemberNotNullAttribute' from assembly 'System.Runtime, Version=6.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a'.
     at System.ModuleHandle.ResolveMethodHandleInternalCore(RuntimeModule module, Int32 methodToken, IntPtr[] typeInstantiationContext, Int32 typeInstCount, IntPtr[] methodInstantiationContext, Int32 methodInstCount)
     at System.ModuleHandle.ResolveMethodHandleInternal(RuntimeModule module, Int32 methodToken, RuntimeTypeHandle[] typeInstantiationContext, RuntimeTypeHandle[] methodInstantiationContext)
     at System.Reflection.CustomAttributeData..ctor(RuntimeModule scope, MetadataToken caCtorToken, ConstArray& blob)
     at System.Reflection.CustomAttributeData.GetCustomAttributes(RuntimeModule module, Int32 tkTarget)
     at System.Reflection.CustomAttributeData.GetCustomAttributesInternal(RuntimeMethodInfo target)
     at System.Reflection.RuntimeMethodInfo.GetCustomAttributesData()
     at System.Reflection.MemberInfo.get_CustomAttributes()
     at Xunit.Sdk.ReflectionMethodInfo.GetCustomAttributes(MethodInfo method, Type attributeType, AttributeUsageAttribute attributeUsage) in C:\Dev\xunit\xunit\src\xunit.execution\Sdk\Reflection\ReflectionMethodInfo.cs:line 93
     at Xunit.Sdk.ReflectionMethodInfo.GetCustomAttributes(MethodInfo method, String assemblyQualifiedAttributeTypeName) in C:\Dev\xunit\xunit\src\xunit.execution\Sdk\Reflection\ReflectionMethodInfo.cs:line 87
     at Xunit.Sdk.ReflectionMethodInfo.GetCustomAttributes(String assemblyQualifiedAttributeTypeName) in C:\Dev\xunit\xunit\src\xunit.execution\Sdk\Reflection\ReflectionMethodInfo.cs:line 80
     at Xunit.Sdk.XunitTestFrameworkDiscoverer.FindTestsForMethod(ITestMethod testMethod, Boolean includeSourceInformation, IMessageBus messageBus, ITestFrameworkDiscoveryOptions discoveryOptions) in C:\Dev\xunit\xunit\src\xunit.execution\Sdk\Frameworks\XunitTestFrameworkDiscoverer.cs:line 84
     at Xunit.Sdk.XunitTestFrameworkDiscoverer.FindTestsForType(ITestClass testClass, Boolean includeSourceInformation, IMessageBus messageBus, ITestFrameworkDiscoveryOptions discoveryOptions) in C:\Dev\xunit\xunit\src\xunit.execution\Sdk\Frameworks\XunitTestFrameworkDiscoverer.cs:line 132
     at Xunit.Sdk.TestFrameworkDiscoverer.FindTestsForTypeAndWrapExceptions(ITestClass testClass, Boolean includeSourceInformation, IMessageBus messageBus, ITestFrameworkDiscoveryOptions discoveryOptions) in C:\Dev\xunit\xunit\src\xunit.execution\Sdk\Frameworks\TestFrameworkDiscoverer.cs:line 156
     Wasm.Build.Tests: Exception during discovery:
  System.TypeLoadException: Could not load type 'System.Diagnostics.CodeAnalysis.MemberNotNullAttribute' from assembly 'System.Runtime, Version=6.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a'.
     at System.ModuleHandle.ResolveMethodHandleInternalCore(RuntimeModule module, Int32 methodToken, IntPtr[] typeInstantiationContext, Int32 typeInstCount, IntPtr[] methodInstantiationContext, Int32 methodInstCount)
     at System.ModuleHandle.ResolveMethodHandleInternal(RuntimeModule module, Int32 methodToken, RuntimeTypeHandle[] typeInstantiationContext, RuntimeTypeHandle[] methodInstantiationContext)
     at System.Reflection.CustomAttributeData..ctor(RuntimeModule scope, MetadataToken caCtorToken, ConstArray& blob)
     at System.Reflection.CustomAttributeData.GetCustomAttributes(RuntimeModule module, Int32 tkTarget)
     at System.Reflection.CustomAttributeData.GetCustomAttributesInternal(RuntimeMethodInfo target)
     at System.Reflection.RuntimeMethodInfo.GetCustomAttributesData()
     at System.Reflection.MemberInfo.get_CustomAttributes()
     at Xunit.Sdk.ReflectionMethodInfo.GetCustomAttributes(MethodInfo method, Type attributeType, AttributeUsageAttribute attributeUsage) in C:\Dev\xunit\xunit\src\xunit.execution\Sdk\Reflection\ReflectionMethodInfo.cs:line 93
     at Xunit.Sdk.ReflectionMethodInfo.GetCustomAttributes(MethodInfo method, String assemblyQualifiedAttributeTypeName) in C:\Dev\xunit\xunit\src\xunit.execution\Sdk\Reflection\ReflectionMethodInfo.cs:line 87
     at Xunit.Sdk.ReflectionMethodInfo.GetCustomAttributes(String assemblyQualifiedAttributeTypeName) in C:\Dev\xunit\xunit\src\xunit.execution\Sdk\Reflection\ReflectionMethodInfo.cs:line 80
     at Xunit.Sdk.XunitTestFrameworkDiscoverer.FindTestsForMethod(ITestMethod testMethod, Boolean includeSourceInformation, IMessageBus messageBus, ITestFrameworkDiscoveryOptions discoveryOptions) in C:\Dev\xunit\xunit\src\xunit.execution\Sdk\Frameworks\XunitTestFrameworkDiscoverer.cs:line 84
     at Xunit.Sdk.XunitTestFrameworkDiscoverer.FindTestsForType(ITestClass testClass, Boolean includeSourceInformation, IMessageBus messageBus, ITestFrameworkDiscoveryOptions discoveryOptions) in C:\Dev\xunit\xunit\src\xunit.execution\Sdk\Frameworks\XunitTestFrameworkDiscoverer.cs:line 132
     at Xunit.Sdk.TestFrameworkDiscoverer.FindTestsForTypeAndWrapExceptions(ITestClass testClass, Boolean includeSourceInformation, IMessageBus messageBus, ITestFrameworkDiscoveryOptions discoveryOptions) in C:\Dev\xunit\xunit\src\xunit.execution\Sdk\Frameworks\TestFrameworkDiscoverer.cs:line 156
     Wasm.Build.Tests: Exception during discovery:
  System.TypeLoadException: Could not load type 'System.Diagnostics.CodeAnalysis.MemberNotNullAttribute' from assembly 'System.Runtime, Version=6.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a'.
     at System.ModuleHandle.ResolveMethodHandleInternalCore(RuntimeModule module, Int32 methodToken, IntPtr[] typeInstantiationContext, Int32 typeInstCount, IntPtr[] methodInstantiationContext, Int32 methodInstCount)
     at System.ModuleHandle.ResolveMethodHandleInternal(RuntimeModule module, Int32 methodToken, RuntimeTypeHandle[] typeInstantiationContext, RuntimeTypeHandle[] methodInstantiationContext)
     at System.Reflection.CustomAttributeData..ctor(RuntimeModule scope, MetadataToken caCtorToken, ConstArray& blob)
     at System.Reflection.CustomAttributeData.GetCustomAttributes(RuntimeModule module, Int32 tkTarget)
     at System.Reflection.CustomAttributeData.GetCustomAttributesInternal(RuntimeMethodInfo target)
     at System.Reflection.RuntimeMethodInfo.GetCustomAttributesData()
     at System.Reflection.MemberInfo.get_CustomAttributes()
     at Xunit.Sdk.ReflectionMethodInfo.GetCustomAttributes(MethodInfo method, Type attributeType, AttributeUsageAttribute attributeUsage) in C:\Dev\xunit\xunit\src\xunit.execution\Sdk\Reflection\ReflectionMethodInfo.cs:line 93
     at Xunit.Sdk.ReflectionMethodInfo.GetCustomAttributes(MethodInfo method, String assemblyQualifiedAttributeTypeName) in C:\Dev\xunit\xunit\src\xunit.execution\Sdk\Reflection\ReflectionMethodInfo.cs:line 87
     at Xunit.Sdk.ReflectionMethodInfo.GetCustomAttributes(String assemblyQualifiedAttributeTypeName) in C:\Dev\xunit\xunit\src\xunit.execution\Sdk\Reflection\ReflectionMethodInfo.cs:line 80
     at Xunit.Sdk.XunitTestFrameworkDiscoverer.FindTestsForMethod(ITestMethod testMethod, Boolean includeSourceInformation, IMessageBus messageBus, ITestFrameworkDiscoveryOptions discoveryOptions) in C:\Dev\xunit\xunit\src\xunit.execution\Sdk\Frameworks\XunitTestFrameworkDiscoverer.cs:line 84
     at Xunit.Sdk.XunitTestFrameworkDiscoverer.FindTestsForType(ITestClass testClass, Boolean includeSourceInformation, IMessageBus messageBus, ITestFrameworkDiscoveryOptions discoveryOptions) in C:\Dev\xunit\xunit\src\xunit.execution\Sdk\Frameworks\XunitTestFrameworkDiscoverer.cs:line 132
     at Xunit.Sdk.TestFrameworkDiscoverer.FindTestsForTypeAndWrapExceptions(ITestClass testClass, Boolean includeSourceInformation, IMessageBus messageBus, ITestFrameworkDiscoveryOptions discoveryOptions) in C:\Dev\xunit\xunit\src\xunit.execution\Sdk\Frameworks\TestFrameworkDiscoverer.cs:line 156
     Wasm.Build.Tests: Exception during discovery:
  System.TypeLoadException: Could not load type 'System.Diagnostics.CodeAnalysis.MemberNotNullAttribute' from assembly 'System.Runtime, Version=6.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a'.
     at System.ModuleHandle.ResolveMethodHandleInternalCore(RuntimeModule module, Int32 methodToken, IntPtr[] typeInstantiationContext, Int32 typeInstCount, IntPtr[] methodInstantiationContext, Int32 methodInstCount)
     at System.ModuleHandle.ResolveMethodHandleInternal(RuntimeModule module, Int32 methodToken, RuntimeTypeHandle[] typeInstantiationContext, RuntimeTypeHandle[] methodInstantiationContext)
     at System.Reflection.CustomAttributeData..ctor(RuntimeModule scope, MetadataToken caCtorToken, ConstArray& blob)
     at System.Reflection.CustomAttributeData.GetCustomAttributes(RuntimeModule module, Int32 tkTarget)
     at System.Reflection.CustomAttributeData.GetCustomAttributesInternal(RuntimeMethodInfo target)
     at System.Reflection.RuntimeMethodInfo.GetCustomAttributesData()
     at System.Reflection.MemberInfo.get_CustomAttributes()
     at Xunit.Sdk.ReflectionMethodInfo.GetCustomAttributes(MethodInfo method, Type attributeType, AttributeUsageAttribute attributeUsage) in C:\Dev\xunit\xunit\src\xunit.execution\Sdk\Reflection\ReflectionMethodInfo.cs:line 93
     at Xunit.Sdk.ReflectionMethodInfo.GetCustomAttributes(MethodInfo method, String assemblyQualifiedAttributeTypeName) in C:\Dev\xunit\xunit\src\xunit.execution\Sdk\Reflection\ReflectionMethodInfo.cs:line 87
     at Xunit.Sdk.ReflectionMethodInfo.GetCustomAttributes(String assemblyQualifiedAttributeTypeName) in C:\Dev\xunit\xunit\src\xunit.execution\Sdk\Reflection\ReflectionMethodInfo.cs:line 80
     at Xunit.Sdk.XunitTestFrameworkDiscoverer.FindTestsForMethod(ITestMethod testMethod, Boolean includeSourceInformation, IMessageBus messageBus, ITestFrameworkDiscoveryOptions discoveryOptions) in C:\Dev\xunit\xunit\src\xunit.execution\Sdk\Frameworks\XunitTestFrameworkDiscoverer.cs:line 84
     at Xunit.Sdk.XunitTestFrameworkDiscoverer.FindTestsForType(ITestClass testClass, Boolean includeSourceInformation, IMessageBus messageBus, ITestFrameworkDiscoveryOptions discoveryOptions) in C:\Dev\xunit\xunit\src\xunit.execution\Sdk\Frameworks\XunitTestFrameworkDiscoverer.cs:line 132
     at Xunit.Sdk.TestFrameworkDiscoverer.FindTestsForTypeAndWrapExceptions(ITestClass testClass, Boolean includeSourceInformation, IMessageBus messageBus, ITestFrameworkDiscoveryOptions discoveryOptions) in C:\Dev\xunit\xunit\src\xunit.execution\Sdk\Frameworks\TestFrameworkDiscoverer.cs:line 156
     Wasm.Build.Tests: Exception during discovery:
  System.TypeLoadException: Could not load type 'System.Diagnostics.CodeAnalysis.MemberNotNullAttribute' from assembly 'System.Runtime, Version=6.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a'.
     at System.ModuleHandle.ResolveMethodHandleInternalCore(RuntimeModule module, Int32 methodToken, IntPtr[] typeInstantiationContext, Int32 typeInstCount, IntPtr[] methodInstantiationContext, Int32 methodInstCount)
     at System.ModuleHandle.ResolveMethodHandleInternal(RuntimeModule module, Int32 methodToken, RuntimeTypeHandle[] typeInstantiationContext, RuntimeTypeHandle[] methodInstantiationContext)
     at System.Reflection.CustomAttributeData..ctor(RuntimeModule scope, MetadataToken caCtorToken, ConstArray& blob)
     at System.Reflection.CustomAttributeData.GetCustomAttributes(RuntimeModule module, Int32 tkTarget)
     at System.Reflection.CustomAttributeData.GetCustomAttributesInternal(RuntimeMethodInfo target)
     at System.Reflection.RuntimeMethodInfo.GetCustomAttributesData()
     at System.Reflection.MemberInfo.get_CustomAttributes()
     at Xunit.Sdk.ReflectionMethodInfo.GetCustomAttributes(MethodInfo method, Type attributeType, AttributeUsageAttribute attributeUsage) in C:\Dev\xunit\xunit\src\xunit.execution\Sdk\Reflection\ReflectionMethodInfo.cs:line 93
     at Xunit.Sdk.ReflectionMethodInfo.GetCustomAttributes(MethodInfo method, String assemblyQualifiedAttributeTypeName) in C:\Dev\xunit\xunit\src\xunit.execution\Sdk\Reflection\ReflectionMethodInfo.cs:line 87
     at Xunit.Sdk.ReflectionMethodInfo.GetCustomAttributes(String assemblyQualifiedAttributeTypeName) in C:\Dev\xunit\xunit\src\xunit.execution\Sdk\Reflection\ReflectionMethodInfo.cs:line 80
     at Xunit.Sdk.XunitTestFrameworkDiscoverer.FindTestsForMethod(ITestMethod testMethod, Boolean includeSourceInformation, IMessageBus messageBus, ITestFrameworkDiscoveryOptions discoveryOptions) in C:\Dev\xunit\xunit\src\xunit.execution\Sdk\Frameworks\XunitTestFrameworkDiscoverer.cs:line 84
     at Xunit.Sdk.XunitTestFrameworkDiscoverer.FindTestsForType(ITestClass testClass, Boolean includeSourceInformation, IMessageBus messageBus, ITestFrameworkDiscoveryOptions discoveryOptions) in C:\Dev\xunit\xunit\src\xunit.execution\Sdk\Frameworks\XunitTestFrameworkDiscoverer.cs:line 132
     at Xunit.Sdk.TestFrameworkDiscoverer.FindTestsForTypeAndWrapExceptions(ITestClass testClass, Boolean includeSourceInformation, IMessageBus messageBus, ITestFrameworkDiscoveryOptions discoveryOptions) in C:\Dev\xunit\xunit\src\xunit.execution\Sdk\Frameworks\TestFrameworkDiscoverer.cs:line 156
    Discovered:  Wasm.Build.Tests (found 14 test cases)
    Starting:    Wasm.Build.Tests (parallel test collections = on, max threads = 48)
      Wasm.Build.Tests.RebuildTests.NoOpRebuild [FAIL]
        System.TypeInitializationException : The type initializer for 'Wasm.Build.Tests.BuildTestBase' threw an exception.
        ---- System.DllNotFoundException : Unable to load DLL 'libSystem.Native' or one of its dependencies: The specified module could not be found. (0x8007007E)
        Stack Trace:
          C:\Users\rodo\git\wa-main\src\tests\BuildWasmApps\Wasm.Build.Tests\BuildTestBase.cs(123,0): at Wasm.Build.Tests.BuildTestBase.ConfigWithAOTData(Boolean aot)
          C:\Users\rodo\git\wa-main\src\tests\BuildWasmApps\Wasm.Build.Tests\InvariantGlobalizationTests.cs(21,0): at Wasm.Build.Tests.InvariantGlobalizationTests.InvariantGlobalizationTestData(Boolean aot, RunHost host)
          ----- Inner Stack Trace -----
             at Interop.Sys.Stat(Byte& path, FileStatus& output)
             at Interop.Sys.Stat(ReadOnlySpan`1 path, FileStatus& output)
             at System.IO.FileSystem.FileExists(ReadOnlySpan`1 fullPath, Int32 fileType, ErrorInfo& errorInfo)
             at System.IO.FileSystem.FileExists(ReadOnlySpan`1 fullPath)
             at System.IO.File.Exists(String path)
          C:\Users\rodo\git\wa-main\src\tests\BuildWasmApps\Wasm.Build.Tests\BuildTestBase.cs(44,0): at Wasm.Build.Tests.BuildTestBase..cctor()
      Wasm.Build.Tests.NativeBuildTests.Relinking_ErrorWhenMissingEMSDK [FAIL]
        System.TypeInitializationException : The type initializer for 'Wasm.Build.Tests.BuildTestBase' threw an exception.
        ---- System.DllNotFoundException : Unable to load DLL 'libSystem.Native' or one of its dependencies: The specified module could not be found. (0x8007007E)
        Stack Trace:
          C:\Users\rodo\git\wa-main\src\tests\BuildWasmApps\Wasm.Build.Tests\BuildTestBase.cs(123,0): at Wasm.Build.Tests.BuildTestBase.ConfigWithAOTData(Boolean aot)
          C:\Users\rodo\git\wa-main\src\tests\BuildWasmApps\Wasm.Build.Tests\InvariantGlobalizationTests.cs(21,0): at Wasm.Build.Tests.InvariantGlobalizationTests.InvariantGlobalizationTestData(Boolean aot, RunHost host)
          ----- Inner Stack Trace -----
             at Interop.Sys.Stat(Byte& path, FileStatus& output)
             at Interop.Sys.Stat(ReadOnlySpan`1 path, FileStatus& output)
             at System.IO.FileSystem.FileExists(ReadOnlySpan`1 fullPath, Int32 fileType, ErrorInfo& errorInfo)
             at System.IO.FileSystem.FileExists(ReadOnlySpan`1 fullPath)
             at System.IO.File.Exists(String path)
          C:\Users\rodo\git\wa-main\src\tests\BuildWasmApps\Wasm.Build.Tests\BuildTestBase.cs(44,0): at Wasm.Build.Tests.BuildTestBase..cctor()
      Wasm.Build.Tests.MainWithArgsTests.TopLevelWithArgs [FAIL]
...
  === TEST EXECUTION SUMMARY ===
     Wasm.Build.Tests  Total: 14, Errors: 5, Failed: 14, Skipped: 0, Time: 0.094s

Author:	radekdoulik
Assignees:	-
Labels:	arch-wasm, area-Build-mono
Milestone:	-

[radical]

Sounds like you might be missing the payload for the build support directory, which gets added at https://github.com/dotnet/runtime/blob/main/src/libraries/sendtohelixhelp.proj#L292-L299 .

• And that depends on $(NeedsToBuildWasmAppsOnHelix), which in this case would become true when Scenario==BuildWasmApps(https://github.com/dotnet/runtime/blob/main/src/libraries/sendtohelixhelp.proj#L53).
• And scenario gets set like (example) https://github.com/dotnet/runtime/blob/main/eng/pipelines/runtime.yml#L297-L298

[radical]

Is there a PR where you are testing this?

[VincentBu]

Failed again in runtime 20210527.100

Failed tests:

net6.0-Browser-Release-wasm-Mono_Release-buildwasmapps-Ubuntu.1804.Amd64.Open

- Wasm.Build.Tests.InvariantGlobalizationTests.AOT_InvariantGlobalization
- Wasm.Build.Tests.InvariantGlobalizationTests.RelinkingWithoutAOT
- Wasm.Build.Tests.MainWithArgsTests.AsyncMainWithArgs
- Wasm.Build.Tests.MainWithArgsTests.NonAsyncMainWithArgs
- Wasm.Build.Tests.MainWithArgsTests.TopLevelWithArgs
- Wasm.Build.Tests.NativeBuildTests.Relinking_ErrorWhenMissingEMSDK
- Wasm.Build.Tests.NativeBuildTests.SimpleNativeBuild
- Wasm.Build.Tests.RebuildTests.NoOpRebuild
- Wasm.Build.Tests.WasmBuildAppTest.AOT_ErrorWhenMissingEMSDK
- Wasm.Build.Tests.WasmBuildAppTest.AsyncMain
- Wasm.Build.Tests.WasmBuildAppTest.Bug49588_RegressionTest_AOT
- Wasm.Build.Tests.WasmBuildAppTest.Bug49588_RegressionTest_NativeRelinking
- Wasm.Build.Tests.WasmBuildAppTest.NonAsyncMain
- Wasm.Build.Tests.WasmBuildAppTest.TopLevelMain

Error message:

System.TypeInitializationException : The type initializer for 'Wasm.Build.Tests.BuildTestBase' threw an exception.
---- System.TypeLoadException : Could not load type 'System.IO.DirectoryInfo' from assembly 'System.Runtime, Version=6.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a'.


Stack trace
   at Wasm.Build.Tests.BuildTestBase.ConfigWithAOTData(Boolean aot) in /_/src/tests/BuildWasmApps/Wasm.Build.Tests/BuildTestBase.cs:line 123
   at Wasm.Build.Tests.InvariantGlobalizationTests.InvariantGlobalizationTestData(Boolean aot, RunHost host) in /_/src/tests/BuildWasmApps/Wasm.Build.Tests/InvariantGlobalizationTests.cs:line 21
----- Inner Stack Trace -----

[radical]

this is a different error. I'm working on it.

[radical]

@VincentBu fyi #53405