[browser][MT] ThreadStartException in System.Net.WebSockets.Client.Tests on WASM

[danmoseley]

warn: 
warn: Unhandled Exception:
warn: System.Threading.ThreadStartException: Thread failed to start.
warn:  ---> System.ExecutionEngineException: mono_thread_platform_create_thread() failed
warn:    --- End of inner exception stack trace ---
warn:    at System.Threading.Thread.ThrowThreadStartException(Exception ex)
warn:    at System.Threading.Thread.StartCore()
warn:    at System.Threading.Thread.Start(Boolean captureContext, Boolean internalThread)
warn:    at System.Threading.Thread.UnsafeStart()
warn:    at System.Threading.WebWorkerEventLoop.StartExitable(Thread thread, Boolean captureContext)
warn:    at System.Threading.PortableThreadPool.WorkerThread.CreateWorkerThread()
warn:    at System.Threading.PortableThreadPool.WorkerThread.MaybeAddWorkingWorker(PortableThreadPool threadPoolInstance)
warn:    at System.Threading.PortableThreadPool.RequestWorker()
warn:    at System.Threading.ThreadPoolWorkQueue.Dispatch()
warn:    at System.Threading.PortableThreadPool.WorkerThread.WorkLoopSemaphoreSuccess(LowLevelLifoAsyncWaitSemaphore semaphore, Object stateObject)
warn:    at System.Threading.LowLevelLifoAsyncWaitSemaphore.InternalAsyncWaitSuccess(LowLevelLifoAsyncWaitSemaphore self, WaitEntry internalWaitEntry)
warn:    at System.Threading.LowLevelLifoAsyncWaitSemaphore.SuccessCallback(IntPtr lifoSemaphore, IntPtr userData)
warn: [2023-10-10T23:38:40.616Z] [ERROR] FATAL UNHANDLED EXCEPTION: System.Threading.ThreadStartException: Thread failed to start.
warn:  ---> System.ExecutionEngineException: mono_thread_platform_create_thread() failed
warn:    --- End of inner exception stack trace ---
warn:    at System.Threading.Thread.ThrowThreadStartException(Exception ex)
warn:    at System.Threading.Thread.StartCore()
warn:    at System.Threading.Thread.Start(Boolean captureContext, Boolean internalThread)
warn:    at System.Threading.Thread.UnsafeStart()
warn:    at System.Threading.WebWorkerEventLoop.StartExitable(Thread thread, Boolean captureContext)
warn:    at System.Threading.PortableThreadPool.WorkerThread.CreateWorkerThread()
warn:    at System.Threading.PortableThreadPool.WorkerThread.MaybeAddWorkingWorker(PortableThreadPool threadPoolInstance)
warn:    at System.Threading.PortableThreadPool.RequestWorker()
warn:    at System.Threading.ThreadPoolWorkQueue.Dispatch()
warn:    at System.Threading.PortableThreadPool.WorkerThread.WorkLoopSemaphoreSuccess(LowLevelLifoAsyncWaitSemaphore semaphore, Object stateObject)
warn:    at System.Threading.LowLevelLifoAsyncWaitSemaphore.InternalAsyncWaitSuccess(LowLevelLifoAsyncWaitSemaphore self, WaitEntry internalWaitEntry)
warn:    at System.Threading.LowLevelLifoAsyncWaitSemaphore.SuccessCallback(IntPtr lifoSemaphore, IntPtr userData)
info: Waiting to flush log messages with a timeout of 120 secs ..
fail: Flushing log messages failed with: System.OperationCanceledException: The operation was canceled.
         at System.Threading.Channels.AsyncOperation`1.GetResult(Int16 token)
         at System.Threading.Channels.ChannelReader`1.ReadAllAsync(CancellationToken cancellationToken)+MoveNext()
         at System.Threading.Channels.ChannelReader`1.ReadAllAsync(CancellationToken cancellationToken)+System.Threading.Tasks.Sources.IValueTaskSource<System.Boolean>.GetResult()
         at Microsoft.DotNet.XHarness.CLI.Commands.Wasm.WasmTestMessagesProcessor.RunAsync(CancellationToken token) in /_/src/Microsoft.DotNet.XHarness.CLI/Commands/WASM/WasmTestMessagesProcessor.cs:line 68
         at Microsoft.DotNet.XHarness.CLI.Commands.Wasm.WasmTestMessagesProcessor.RunAsync(CancellationToken token) in /_/src/Microsoft.DotNet.XHarness.CLI/Commands/WASM/WasmTestMessagesProcessor.cs:line 68
         at Microsoft.DotNet.XHarness.CLI.Commands.Wasm.WasmTestMessagesProcessor.CompleteAndFlushAsync(Nullable`1 timeout) in /_/src/Microsoft.DotNet.XHarness.CLI/Commands/WASM/WasmTestMessagesProcessor.cs:line 140. Ignoring.
fail: Tests timed out. Killing driver service pid 79

Build Information

Build: https://helixre107v0xdcypoyl9e7f.blob.core.windows.net/dotnet-runtime-refs-pull-92682-merge-d089dcadfcc74738af/WasmTestOnBrowser-System.Net.WebSockets.Client.Tests/1/console.7cb91235.log?helixlogtype=result
Leg Name: Console log: 'WasmTestOnBrowser-System.Net.WebSockets.Client.Tests' from job d089dcad-fcc7-4738-afd1-f43163be9d35 (ubuntu.2204.amd64.open.rt) using docker image mcr.microsoft.com/dotnet-buildtools/prereqs:ubuntu-22.04-helix-webassembly on a002M59

Error Message

{ 
    "ErrorMessage": "System.ExecutionEngineException: mono_thread_platform_create_thread() failed",
    "BuildRetry": true,
    "ErrorPattern": "",
    "ExcludeConsoleLog": false
} 

Tagging subscribers to this area: @mangod9
See info in area-owners.md if you want to be subscribed.

Issue Details

---

warn: 
warn: Unhandled Exception:
warn: System.Threading.ThreadStartException: Thread failed to start.
warn:  ---> System.ExecutionEngineException: mono_thread_platform_create_thread() failed
warn:    --- End of inner exception stack trace ---
warn:    at System.Threading.Thread.ThrowThreadStartException(Exception ex)
warn:    at System.Threading.Thread.StartCore()
warn:    at System.Threading.Thread.Start(Boolean captureContext, Boolean internalThread)
warn:    at System.Threading.Thread.UnsafeStart()
warn:    at System.Threading.WebWorkerEventLoop.StartExitable(Thread thread, Boolean captureContext)
warn:    at System.Threading.PortableThreadPool.WorkerThread.CreateWorkerThread()
warn:    at System.Threading.PortableThreadPool.WorkerThread.MaybeAddWorkingWorker(PortableThreadPool threadPoolInstance)
warn:    at System.Threading.PortableThreadPool.RequestWorker()
warn:    at System.Threading.ThreadPoolWorkQueue.Dispatch()
warn:    at System.Threading.PortableThreadPool.WorkerThread.WorkLoopSemaphoreSuccess(LowLevelLifoAsyncWaitSemaphore semaphore, Object stateObject)
warn:    at System.Threading.LowLevelLifoAsyncWaitSemaphore.InternalAsyncWaitSuccess(LowLevelLifoAsyncWaitSemaphore self, WaitEntry internalWaitEntry)
warn:    at System.Threading.LowLevelLifoAsyncWaitSemaphore.SuccessCallback(IntPtr lifoSemaphore, IntPtr userData)
warn: [2023-10-10T23:38:40.616Z] [ERROR] FATAL UNHANDLED EXCEPTION: System.Threading.ThreadStartException: Thread failed to start.
warn:  ---> System.ExecutionEngineException: mono_thread_platform_create_thread() failed
warn:    --- End of inner exception stack trace ---
warn:    at System.Threading.Thread.ThrowThreadStartException(Exception ex)
warn:    at System.Threading.Thread.StartCore()
warn:    at System.Threading.Thread.Start(Boolean captureContext, Boolean internalThread)
warn:    at System.Threading.Thread.UnsafeStart()
warn:    at System.Threading.WebWorkerEventLoop.StartExitable(Thread thread, Boolean captureContext)
warn:    at System.Threading.PortableThreadPool.WorkerThread.CreateWorkerThread()
warn:    at System.Threading.PortableThreadPool.WorkerThread.MaybeAddWorkingWorker(PortableThreadPool threadPoolInstance)
warn:    at System.Threading.PortableThreadPool.RequestWorker()
warn:    at System.Threading.ThreadPoolWorkQueue.Dispatch()
warn:    at System.Threading.PortableThreadPool.WorkerThread.WorkLoopSemaphoreSuccess(LowLevelLifoAsyncWaitSemaphore semaphore, Object stateObject)
warn:    at System.Threading.LowLevelLifoAsyncWaitSemaphore.InternalAsyncWaitSuccess(LowLevelLifoAsyncWaitSemaphore self, WaitEntry internalWaitEntry)
warn:    at System.Threading.LowLevelLifoAsyncWaitSemaphore.SuccessCallback(IntPtr lifoSemaphore, IntPtr userData)
info: Waiting to flush log messages with a timeout of 120 secs ..
fail: Flushing log messages failed with: System.OperationCanceledException: The operation was canceled.
         at System.Threading.Channels.AsyncOperation`1.GetResult(Int16 token)
         at System.Threading.Channels.ChannelReader`1.ReadAllAsync(CancellationToken cancellationToken)+MoveNext()
         at System.Threading.Channels.ChannelReader`1.ReadAllAsync(CancellationToken cancellationToken)+System.Threading.Tasks.Sources.IValueTaskSource<System.Boolean>.GetResult()
         at Microsoft.DotNet.XHarness.CLI.Commands.Wasm.WasmTestMessagesProcessor.RunAsync(CancellationToken token) in /_/src/Microsoft.DotNet.XHarness.CLI/Commands/WASM/WasmTestMessagesProcessor.cs:line 68
         at Microsoft.DotNet.XHarness.CLI.Commands.Wasm.WasmTestMessagesProcessor.RunAsync(CancellationToken token) in /_/src/Microsoft.DotNet.XHarness.CLI/Commands/WASM/WasmTestMessagesProcessor.cs:line 68
         at Microsoft.DotNet.XHarness.CLI.Commands.Wasm.WasmTestMessagesProcessor.CompleteAndFlushAsync(Nullable`1 timeout) in /_/src/Microsoft.DotNet.XHarness.CLI/Commands/WASM/WasmTestMessagesProcessor.cs:line 140. Ignoring.
fail: Tests timed out. Killing driver service pid 79

Build Information

Build: https://helixre107v0xdcypoyl9e7f.blob.core.windows.net/dotnet-runtime-refs-pull-92682-merge-d089dcadfcc74738af/WasmTestOnBrowser-System.Net.WebSockets.Client.Tests/1/console.7cb91235.log?helixlogtype=result
Leg Name: Console log: 'WasmTestOnBrowser-System.Net.WebSockets.Client.Tests' from job d089dcad-fcc7-4738-afd1-f43163be9d35 (ubuntu.2204.amd64.open.rt) using docker image mcr.microsoft.com/dotnet-buildtools/prereqs:ubuntu-22.04-helix-webassembly on a002M59

Error Message

{ 
    "ErrorMessage": "System.ExecutionEngineException: mono_thread_platform_create_thread() failed",
    "BuildRetry": true,
    "ErrorPattern": "",
    "ExcludeConsoleLog": false
} 

Author:	danmoseley
Assignees:	-
Labels:	area-System.Threading
Milestone:	-

[danmoseley]

@ilonatommy there are several issues with other tests but similar failure:
https://github.com/dotnet/runtime/issues?q=is%3Aissue+is%3Aopen+ThrowThreadStartException

none of them have a known issues blob so I made this one fairly generic.

Tagging subscribers to 'arch-wasm': @lewing
See info in area-owners.md if you want to be subscribed.

Issue Details

---

warn: 
warn: Unhandled Exception:
warn: System.Threading.ThreadStartException: Thread failed to start.
warn:  ---> System.ExecutionEngineException: mono_thread_platform_create_thread() failed
warn:    --- End of inner exception stack trace ---
warn:    at System.Threading.Thread.ThrowThreadStartException(Exception ex)
warn:    at System.Threading.Thread.StartCore()
warn:    at System.Threading.Thread.Start(Boolean captureContext, Boolean internalThread)
warn:    at System.Threading.Thread.UnsafeStart()
warn:    at System.Threading.WebWorkerEventLoop.StartExitable(Thread thread, Boolean captureContext)
warn:    at System.Threading.PortableThreadPool.WorkerThread.CreateWorkerThread()
warn:    at System.Threading.PortableThreadPool.WorkerThread.MaybeAddWorkingWorker(PortableThreadPool threadPoolInstance)
warn:    at System.Threading.PortableThreadPool.RequestWorker()
warn:    at System.Threading.ThreadPoolWorkQueue.Dispatch()
warn:    at System.Threading.PortableThreadPool.WorkerThread.WorkLoopSemaphoreSuccess(LowLevelLifoAsyncWaitSemaphore semaphore, Object stateObject)
warn:    at System.Threading.LowLevelLifoAsyncWaitSemaphore.InternalAsyncWaitSuccess(LowLevelLifoAsyncWaitSemaphore self, WaitEntry internalWaitEntry)
warn:    at System.Threading.LowLevelLifoAsyncWaitSemaphore.SuccessCallback(IntPtr lifoSemaphore, IntPtr userData)
warn: [2023-10-10T23:38:40.616Z] [ERROR] FATAL UNHANDLED EXCEPTION: System.Threading.ThreadStartException: Thread failed to start.
warn:  ---> System.ExecutionEngineException: mono_thread_platform_create_thread() failed
warn:    --- End of inner exception stack trace ---
warn:    at System.Threading.Thread.ThrowThreadStartException(Exception ex)
warn:    at System.Threading.Thread.StartCore()
warn:    at System.Threading.Thread.Start(Boolean captureContext, Boolean internalThread)
warn:    at System.Threading.Thread.UnsafeStart()
warn:    at System.Threading.WebWorkerEventLoop.StartExitable(Thread thread, Boolean captureContext)
warn:    at System.Threading.PortableThreadPool.WorkerThread.CreateWorkerThread()
warn:    at System.Threading.PortableThreadPool.WorkerThread.MaybeAddWorkingWorker(PortableThreadPool threadPoolInstance)
warn:    at System.Threading.PortableThreadPool.RequestWorker()
warn:    at System.Threading.ThreadPoolWorkQueue.Dispatch()
warn:    at System.Threading.PortableThreadPool.WorkerThread.WorkLoopSemaphoreSuccess(LowLevelLifoAsyncWaitSemaphore semaphore, Object stateObject)
warn:    at System.Threading.LowLevelLifoAsyncWaitSemaphore.InternalAsyncWaitSuccess(LowLevelLifoAsyncWaitSemaphore self, WaitEntry internalWaitEntry)
warn:    at System.Threading.LowLevelLifoAsyncWaitSemaphore.SuccessCallback(IntPtr lifoSemaphore, IntPtr userData)
info: Waiting to flush log messages with a timeout of 120 secs ..
fail: Flushing log messages failed with: System.OperationCanceledException: The operation was canceled.
         at System.Threading.Channels.AsyncOperation`1.GetResult(Int16 token)
         at System.Threading.Channels.ChannelReader`1.ReadAllAsync(CancellationToken cancellationToken)+MoveNext()
         at System.Threading.Channels.ChannelReader`1.ReadAllAsync(CancellationToken cancellationToken)+System.Threading.Tasks.Sources.IValueTaskSource<System.Boolean>.GetResult()
         at Microsoft.DotNet.XHarness.CLI.Commands.Wasm.WasmTestMessagesProcessor.RunAsync(CancellationToken token) in /_/src/Microsoft.DotNet.XHarness.CLI/Commands/WASM/WasmTestMessagesProcessor.cs:line 68
         at Microsoft.DotNet.XHarness.CLI.Commands.Wasm.WasmTestMessagesProcessor.RunAsync(CancellationToken token) in /_/src/Microsoft.DotNet.XHarness.CLI/Commands/WASM/WasmTestMessagesProcessor.cs:line 68
         at Microsoft.DotNet.XHarness.CLI.Commands.Wasm.WasmTestMessagesProcessor.CompleteAndFlushAsync(Nullable`1 timeout) in /_/src/Microsoft.DotNet.XHarness.CLI/Commands/WASM/WasmTestMessagesProcessor.cs:line 140. Ignoring.
fail: Tests timed out. Killing driver service pid 79

Build Information

Build: https://helixre107v0xdcypoyl9e7f.blob.core.windows.net/dotnet-runtime-refs-pull-92682-merge-d089dcadfcc74738af/WasmTestOnBrowser-System.Net.WebSockets.Client.Tests/1/console.7cb91235.log?helixlogtype=result
Leg Name: Console log: 'WasmTestOnBrowser-System.Net.WebSockets.Client.Tests' from job d089dcad-fcc7-4738-afd1-f43163be9d35 (ubuntu.2204.amd64.open.rt) using docker image mcr.microsoft.com/dotnet-buildtools/prereqs:ubuntu-22.04-helix-webassembly on a002M59

Error Message

{ 
    "ErrorMessage": "System.ExecutionEngineException: mono_thread_platform_create_thread() failed",
    "BuildRetry": true,
    "ErrorPattern": "",
    "ExcludeConsoleLog": false
} 

Author:	danmoseley
Assignees:	-
Labels:	arch-wasm, untriaged, area-VM-threading-mono
Milestone:	-

[lambdageek]

There's a setting in the threadpool for the maximum number of threads that it is allowed to start that can be set in runtimeconfig.json: https://learn.microsoft.com/en-us/dotnet/core/runtime-config/threading#maximum-threads. we should probably set it to be equal to the max number of threads that we tell emscripten to allow (minus 2-3 - the runtime consumes a bunch of threads for its own uses)