[wasm-mt] Add MessageChannel between browser thread and new pthreads (#70908)

Add a mechanism for creating a communication channel between the browser thread and new pthreads (running in webworkers - note, Emscripten may recycle workers after a pthread exits).

This is done by adding our own event listener to the webworker (in the main thread) and to globalThis (in the worker). This conflicts with emscripten's message handlers (although it's considered a bug by Emscripten upstream that their event handler doesn't ignore unrecognized messages).

One potential problem here is that for any communication to happen, the worker must service its event loop. If it's just busy running a loop in wasm, it might never handle the messages. On the other hand, posting messages back to main should work.

Once we have our message handlers in place, the rest is straightforward, the worker creates a MessageChannel and transfers one of the ports to the browser thread.

With the browser-to-pthread channel established, we can build up cross-thread channels by asking the main thread to transfer ports on our behalf. This part isn't done yet.

---

Additionally in the worker, create an `EventTarget` that fires `dotnet:pthread:created` and `dotnet:pthread:attached` events whenever Emscripten begins running a new thread on one of its workers, and whenever that worker attaches to Mono.  This lets runtime subsystems be notified on the JS side whenever threads come into existence or may potentially begin to run managed code.

---

Also re-organizes our `tsconfig.json` into `tsconfig.shared.json` (common flags), `tsconfig.worker.json` (uses the `esnext` and `worker` libs, so VS Code doesn't offer DOM completions and types, for example), and `tsconfig.json` (uses the `esnext` and `dom` libs).  Subsystems with their own subdirectories (like `pthreads/worker` `pthreads/browser`, etc) can use the `tsconfig` `extends` property to include the appropriate root-directory config.

---

* outline of a dedicated channel between JS main thread and pthreads

* add JS entrypoints for pthread-channel

* wire up the MessageChannel to native thread attach

* debug printfs etc

* split up into pthread-channel module into worker, browser and shared

* pthreads modules

* add ENVIRONMENT_IS_PTHREAD; mono_wasm_pthread_worker_init

* Fixup names; call MessagePort.start; remove printfs

* remove whitespace and extra printfs

* Exclude threading exports in non-threaded runtime

   Use the `USE_THREADS` emscripten library ifdef to prevent the entrypoints from being saved.

   Use a new `MonoWasmThreads` rollup constant to remove JS roots.

   Verified that a Release build single-threaded dotnet.js doesn't include any of the new pthread support code

* Add replacement for PThread.loadWasmModuleToWorker

   This will allow us to install a message handler when the worker is created, before it has any pthreads assigned to it.

   We can therefore simplify how we set up our own MessageChannel to simply send a single event from the worker to the browser thread, instead of having to lazily install the event handler on the main thread by queueing async work to the browser thread.

* Simplify the dedicated channel creation

   now that we can add a message handler to a worker when Emscripten creates it, skip the complicated atomic notification process

* Don't forget the GC transition out to JS

* fix browser-eventpipe default import

* move mono_threads_wasm_on_thread_attached later in register_thread

   Actually attach the thread to the runtime (so that GC transitions work) before calling out to JS

* also fix default import in browser-mt-eventpipe

* Add replacement for Module.PThread.threadInit

   Use it to call mono_wasm_pthread_on_pthread_created

   Rename the previous callback to mono_wasm_pthread_on_pthread_attached - it gets called but it's unused.

   This is enough to get the diagnostic sever worker started up and pinging.

* Cleanup mono_wasm_pthread_on_pthread_created

* Share tsconfig parts using "extends" property

* Use an EventTarget and custom events for worker thread lifecycle notifications

* pass portToMain in ThreadEvents; update README

   this lets pthread lifecycle event handlers post messages and setup listeners on the message port back to the main browser thread.

   Also update the README to describe the current design

* make pthread/worker/events friendlier to tree shaking and node

* another approach to tree shaking

   rollup doesn't understand fallthru.  In the following (which is what `mono_assert` ammounts to) it retains `C`:

   ```
      if (condition)
        throw new Error (...);
      // fallthru
      return new C();
   ```
   
   Solution is to use an if-then-else
   
   ```
      if (condition)
        throw new Error (...);
      else
        return new C(); // C is not retained if 'condition' is false
   ```

* fix annoying VSCode ESLint toast

   Cannot read property 'loc' of undefined

  See microsoft/vscode-eslint#1149

   and a proposed workaround in eslint/eslint#14538 (comment)

* Add Event and EventTarget polyfill for v8

* fix whitespace and comments

Author: lambdageek

src/mono/mono/component/event_pipe-stub.c

@@ -6,6 +6,9 @@
 #include "mono/component/event_pipe.h"
 #include "mono/component/event_pipe-wasm.h"
 #include "mono/metadata/components.h"
+#ifdef HOST_WASM
+#include <emscripten/emscripten.h>
+#endif
 
 static EventPipeSessionID _dummy_session_id;
 

src/mono/mono/component/event_pipe.c

@@ -13,6 +13,9 @@
 #include <eventpipe/ep-event-instance.h>
 #include <eventpipe/ep-session.h>
 
+#ifdef HOST_WASM
+#include <emscripten/emscripten.h>
+#endif
 
 extern void ep_rt_mono_component_init (void);
 static bool _event_pipe_component_inited = false;

src/mono/mono/utils/mono-threads-wasm.c

@@ -411,6 +411,29 @@ mono_threads_platform_stw_defer_initial_suspend (MonoThreadInfo *info)
 	return mono_native_thread_id_equals (mono_thread_info_get_tid (info), mono_threads_wasm_browser_thread_tid ());
 }
 
+#ifndef DISABLE_THREADS
+extern void
+mono_wasm_pthread_on_pthread_attached (gpointer pthread_id);
+#endif
+
+void
+mono_threads_wasm_on_thread_attached (void)
+{
+#ifdef DISABLE_THREADS
+	return;
+#else
+	if (mono_threads_wasm_is_browser_thread ()) {
+		return;
+	}
+	// Notify JS that the pthread attachd to Mono
+	pthread_t id = pthread_self ();
+	MONO_ENTER_GC_SAFE;
+	mono_wasm_pthread_on_pthread_attached (id);
+	MONO_EXIT_GC_SAFE;
+#endif
+}
+
+
 #ifndef DISABLE_THREADS
 void
 mono_threads_wasm_async_run_in_main_thread (void (*func) (void))
@@ -423,6 +446,14 @@ mono_threads_wasm_async_run_in_main_thread_vi (void (*func) (gpointer), gpointer
 {
 	emscripten_async_run_in_main_runtime_thread (EM_FUNC_SIG_VI, func, user_data);
 }
+
+void
+mono_threads_wasm_async_run_in_main_thread_vii (void (*func) (gpointer, gpointer), gpointer user_data1, gpointer user_data2)
+{
+	emscripten_async_run_in_main_runtime_thread (EM_FUNC_SIG_VII, func, user_data1, user_data2);
+}
+
+
 #endif /* DISABLE_THREADS */
 
 #endif /* HOST_BROWSER */

src/mono/mono/utils/mono-threads-wasm.h

@@ -39,8 +39,15 @@ mono_threads_wasm_async_run_in_main_thread (void (*func) (void));
  */
 void
 mono_threads_wasm_async_run_in_main_thread_vi (void (*func)(gpointer), gpointer user_data);
+
+void
+mono_threads_wasm_async_run_in_main_thread_vii (void (*func)(gpointer, gpointer), gpointer user_data1, gpointer user_data2);
 #endif /* DISABLE_THREADS */
 
+// Called from register_thread when a pthread attaches to the runtime
+void
+mono_threads_wasm_on_thread_attached (void);
+
 #endif /* HOST_WASM*/
 
 #endif /* __MONO_THREADS_WASM_H__ */

src/mono/mono/utils/mono-threads.c

@@ -31,6 +31,7 @@
 #include <mono/utils/mono-coop-semaphore.h>
 #include <mono/utils/mono-threads-coop.h>
 #include <mono/utils/mono-threads-debug.h>
+#include <mono/utils/mono-threads-wasm.h>
 #include <mono/utils/os-event.h>
 #include <mono/utils/w32api.h>
 #include <glib.h>
@@ -552,6 +553,10 @@ register_thread (MonoThreadInfo *info)
 	g_assert (result);
 	mono_thread_info_suspend_unlock ();
 
+#ifdef HOST_BROWSER
+	mono_threads_wasm_on_thread_attached ();
+#endif
+
 	return TRUE;
 }
 

src/mono/sample/wasm/browser-eventpipe/Program.cs

@@ -93,9 +93,23 @@ private static long recursiveFib (int n)
             return recursiveFib (n - 1) + recursiveFib (n - 2);
         }
 
+#if false
+            // dead code to prove that starting user threads isn't possible on the perftracing runtime
+            public static void Meth() {
+                    Thread.Sleep (500);
+                    while (!GetCancellationToken().IsCancellationRequested) {
+                            Console.WriteLine ("ping");
+                            Thread.Sleep (500);
+                    }
+            }
+#endif
+
         public static async Task<double> StartAsyncWork(int N)
         {
             CancellationToken ct = GetCancellationToken();
+#if false
+            new Thread(new ThreadStart(Meth)).Start();
+#endif
             await Task.Delay(1);
             long b;
             WasmHelloEventSource.Instance.NewCallsCounter();

src/mono/sample/wasm/browser-eventpipe/Wasm.Browser.EventPipe.Sample.csproj

@@ -9,7 +9,7 @@
     <GenerateRunScriptForSample Condition="'$(ArchiveTests)' == 'true'">true</GenerateRunScriptForSample>
     <RunScriptCommand>$(ExecXHarnessCmd) wasm test-browser  --app=. --browser=Chrome $(XHarnessBrowserPathArg) --html-file=index.html --output-directory=$(XHarnessOutput) -- $(MSBuildProjectName).dll</RunScriptCommand>
     <FeatureWasmPerfTracing>true</FeatureWasmPerfTracing>
-    <WasmEnablePerfTracing>true</WasmEnablePerfTracing>
+    <FeatureWasmThreads Condition="false">true</FeatureWasmThreads>
     <NoWarn>CA2007</NoWarn> <!-- consider ConfigureAwait() -->
   </PropertyGroup>
 

src/mono/sample/wasm/browser-eventpipe/main.js

@@ -1,4 +1,4 @@
-import { createDotnetRuntime } from "./dotnet.js";
+import createDotnetRuntime from "./dotnet.js";
 
 function downloadData(dataURL, filename) {
     // make an `<a download="filename" href="data:..."/>` link and click on it to trigger a download with the given name

src/mono/sample/wasm/browser-mt-eventpipe/main.js

@@ -1,4 +1,4 @@
-import { createDotnetRuntime } from "./dotnet.js";
+import createDotnetRuntime from "./dotnet.js";
 
 function wasm_exit(exit_code, reason) {
     /* Set result in a tests_done element, to be read by xharness in runonly CI test */

src/mono/wasm/runtime/.eslintrc.cjs

@@ -29,7 +29,10 @@ module.exports = {
         "indent": [
             "error",
             4,
-            { SwitchCase: 1 }
+            {
+                SwitchCase: 1,
+                "ignoredNodes": ["VariableDeclaration[declarations.length=0]"] // fixes https://github.com/microsoft/vscode-eslint/issues/1149
+            }
         ],
         "linebreak-style": "off",
         "quotes": [

src/mono/wasm/runtime/cjs/dotnet.cjs.lib.js

@@ -4,22 +4,39 @@
 
 "use strict";
 
+#if USE_PTHREADS
+const usePThreads = `true`;
+const isPThread = `ENVIRONMENT_IS_PTHREAD`;
+#else
+const usePThreads = `false`;
+const isPThread = `false`;
+#endif
+
 const DotnetSupportLib = {
     $DOTNET: {},
     // these lines will be placed early on emscripten runtime creation, passing import and export objects into __dotnet_runtime IFFE
     // we replace implementation of readAsync and fetch
     // replacement of require is there for consistency with ES6 code
     $DOTNET__postset: `
-let __dotnet_replacements = {readAsync, fetch: globalThis.fetch, require, updateGlobalBufferAndViews};
+let __dotnet_replacement_PThread = ${usePThreads} ? {} : undefined;
+if (${usePThreads}) {
+    __dotnet_replacement_PThread.loadWasmModuleToWorker = PThread.loadWasmModuleToWorker;
+    __dotnet_replacement_PThread.threadInit = PThread.threadInit;
+}
+let __dotnet_replacements = {readAsync, fetch: globalThis.fetch, require, updateGlobalBufferAndViews, pthreadReplacements: __dotnet_replacement_PThread};
 let __dotnet_exportedAPI = __dotnet_runtime.__initializeImportsAndExports(
-    { isESM:false, isGlobal:ENVIRONMENT_IS_GLOBAL, isNode:ENVIRONMENT_IS_NODE, isWorker:ENVIRONMENT_IS_WORKER, isShell:ENVIRONMENT_IS_SHELL, isWeb:ENVIRONMENT_IS_WEB, locateFile, quit_, ExitStatus, requirePromise:Promise.resolve(require)},
+    { isESM:false, isGlobal:ENVIRONMENT_IS_GLOBAL, isNode:ENVIRONMENT_IS_NODE, isWorker:ENVIRONMENT_IS_WORKER, isShell:ENVIRONMENT_IS_SHELL, isWeb:ENVIRONMENT_IS_WEB, isPThread:${isPThread}, locateFile, quit_, ExitStatus, requirePromise:Promise.resolve(require)},
     { mono:MONO, binding:BINDING, internal:INTERNAL, module:Module, marshaled_exports: EXPORTS, marshaled_imports: IMPORTS  },
     __dotnet_replacements);
 updateGlobalBufferAndViews = __dotnet_replacements.updateGlobalBufferAndViews;
 readAsync = __dotnet_replacements.readAsync;
 var fetch = __dotnet_replacements.fetch;
 require = __dotnet_replacements.requireOut;
 var noExitRuntime = __dotnet_replacements.noExitRuntime;
+if (${usePThreads}) {
+    PThread.loadWasmModuleToWorker = __dotnet_replacements.pthreadReplacements.loadWasmModuleToWorker;
+    PThread.threadInit = __dotnet_replacements.pthreadReplacements.threadInit;
+}
 `,
 };
 
@@ -73,6 +90,11 @@ const linked_functions = [
     "dotnet_browser_can_use_subtle_crypto_impl",
     "dotnet_browser_simple_digest_hash",
     "dotnet_browser_sign",
+
+    /// mono-threads-wasm.c
+    #if USE_PTHREADS
+    "mono_wasm_pthread_on_pthread_attached",
+    #endif
 ];
 
 // -- this javascript file is evaluated by emcc during compilation! --

src/mono/wasm/runtime/es6/dotnet.es6.lib.js

@@ -4,6 +4,14 @@
 
 "use strict";
 
+#if USE_PTHREADS
+const usePThreads = `true`;
+const isPThread = `ENVIRONMENT_IS_PTHREAD`;
+#else
+const usePThreads = `false`;
+const isPThread = `false`;
+#endif
+
 const DotnetSupportLib = {
     $DOTNET: {},
     // this line will be placed early on emscripten runtime creation, passing import and export objects into __dotnet_runtime IFFE
@@ -14,7 +22,12 @@ const DotnetSupportLib = {
     // Emscripten's getBinaryPromise is not async for NodeJs, but we would like to have it async, so we replace it.
     // We also replace implementation of readAsync and fetch
     $DOTNET__postset: `
-let __dotnet_replacements = {readAsync, fetch: globalThis.fetch, require, updateGlobalBufferAndViews};
+let __dotnet_replacement_PThread = ${usePThreads} ? {} : undefined;
+if (${usePThreads}) {
+    __dotnet_replacement_PThread.loadWasmModuleToWorker = PThread.loadWasmModuleToWorker;
+    __dotnet_replacement_PThread.threadInit = PThread.threadInit;
+}
+let __dotnet_replacements = {readAsync, fetch: globalThis.fetch, require, updateGlobalBufferAndViews, pthreadReplacements: __dotnet_replacement_PThread};
 if (ENVIRONMENT_IS_NODE) {
     __dotnet_replacements.requirePromise = import(/* webpackIgnore: true */'module').then(mod => {
         const require = mod.createRequire(import.meta.url);
@@ -49,14 +62,18 @@ if (ENVIRONMENT_IS_NODE) {
     }
 }
 let __dotnet_exportedAPI = __dotnet_runtime.__initializeImportsAndExports(
-    { isESM:true, isGlobal:false, isNode:ENVIRONMENT_IS_NODE, isWorker:ENVIRONMENT_IS_WORKER, isShell:ENVIRONMENT_IS_SHELL, isWeb:ENVIRONMENT_IS_WEB, locateFile, quit_, ExitStatus, requirePromise:__dotnet_replacements.requirePromise },
+    { isESM:true, isGlobal:false, isNode:ENVIRONMENT_IS_NODE, isWorker:ENVIRONMENT_IS_WORKER, isShell:ENVIRONMENT_IS_SHELL, isWeb:ENVIRONMENT_IS_WEB, isPThread:${isPThread}, locateFile, quit_, ExitStatus, requirePromise:__dotnet_replacements.requirePromise },
     { mono:MONO, binding:BINDING, internal:INTERNAL, module:Module, marshaled_exports: EXPORTS, marshaled_imports: IMPORTS },
     __dotnet_replacements);
 updateGlobalBufferAndViews = __dotnet_replacements.updateGlobalBufferAndViews;
 readAsync = __dotnet_replacements.readAsync;
 var fetch = __dotnet_replacements.fetch;
 require = __dotnet_replacements.requireOut;
 var noExitRuntime = __dotnet_replacements.noExitRuntime;
+if (${usePThreads}) {
+    PThread.loadWasmModuleToWorker = __dotnet_replacements.pthreadReplacements.loadWasmModuleToWorker;
+    PThread.threadInit = __dotnet_replacements.pthreadReplacements.threadInit;
+}
 `,
 };
 
@@ -110,6 +127,11 @@ const linked_functions = [
     "dotnet_browser_can_use_subtle_crypto_impl",
     "dotnet_browser_simple_digest_hash",
     "dotnet_browser_sign",
+
+    /// mono-threads-wasm.c
+    #if USE_PTHREADS
+    "mono_wasm_pthread_on_pthread_attached",
+    #endif
 ];
 
 // -- this javascript file is evaluated by emcc during compilation! --

src/mono/wasm/runtime/exports.ts

@@ -3,6 +3,7 @@
 
 import ProductVersion from "consts:productVersion";
 import Configuration from "consts:configuration";
+import MonoWasmThreads from "consts:monoWasmThreads";
 
 import {
     mono_wasm_new_root, mono_wasm_release_roots, mono_wasm_new_external_root,
@@ -74,6 +75,8 @@ import {
 } from "./crypto-worker";
 import { mono_wasm_cancel_promise_ref } from "./cancelable-promise";
 import { mono_wasm_web_socket_open_ref, mono_wasm_web_socket_send, mono_wasm_web_socket_receive, mono_wasm_web_socket_close_ref, mono_wasm_web_socket_abort } from "./web-socket";
+import { mono_wasm_pthread_on_pthread_attached, afterThreadInit } from "./pthreads/worker";
+import { afterLoadWasmModuleToWorker } from "./pthreads/browser";
 
 const MONO = {
     // current "public" MONO API
@@ -184,14 +187,20 @@ export type BINDINGType = typeof BINDING;
 
 let exportedAPI: DotnetPublicAPI;
 
+// We need to replace some of the methods in the Emscripten PThreads support with our own
+type PThreadReplacements = {
+    loadWasmModuleToWorker: Function,
+    threadInit: Function
+}
+
 // this is executed early during load of emscripten runtime
 // it exports methods to global objects MONO, BINDING and Module in backward compatible way
 // At runtime this will be referred to as 'createDotnetRuntime'
 // eslint-disable-next-line @typescript-eslint/explicit-module-boundary-types
 function initializeImportsAndExports(
-    imports: { isESM: boolean, isGlobal: boolean, isNode: boolean, isWorker: boolean, isShell: boolean, isWeb: boolean, locateFile: Function, quit_: Function, ExitStatus: ExitStatusError, requirePromise: Promise<Function> },
+    imports: { isESM: boolean, isGlobal: boolean, isNode: boolean, isWorker: boolean, isShell: boolean, isWeb: boolean, isPThread: boolean, locateFile: Function, quit_: Function, ExitStatus: ExitStatusError, requirePromise: Promise<Function> },
     exports: { mono: any, binding: any, internal: any, module: any, marshaled_exports: any, marshaled_imports: any },
-    replacements: { fetch: any, readAsync: any, require: any, requireOut: any, noExitRuntime: boolean, updateGlobalBufferAndViews: Function },
+    replacements: { fetch: any, readAsync: any, require: any, requireOut: any, noExitRuntime: boolean, updateGlobalBufferAndViews: Function, pthreadReplacements: PThreadReplacements | undefined | null },
 ): DotnetPublicAPI {
     const module = exports.module as DotnetModule;
     const globalThisAny = globalThis as any;
@@ -258,6 +267,19 @@ function initializeImportsAndExports(
 
     replacements.noExitRuntime = ENVIRONMENT_IS_WEB;
 
+    if (replacements.pthreadReplacements) {
+        const originalLoadWasmModuleToWorker = replacements.pthreadReplacements.loadWasmModuleToWorker;
+        replacements.pthreadReplacements.loadWasmModuleToWorker = (worker: Worker, onFinishedLoading: Function): void => {
+            originalLoadWasmModuleToWorker(worker, onFinishedLoading);
+            afterLoadWasmModuleToWorker(worker);
+        };
+        const originalThreadInit = replacements.pthreadReplacements.threadInit;
+        replacements.pthreadReplacements.threadInit = (): void => {
+            originalThreadInit();
+            afterThreadInit();
+        };
+    }
+
     if (typeof module.disableDotnet6Compatibility === "undefined") {
         module.disableDotnet6Compatibility = imports.isESM;
     }
@@ -328,6 +350,13 @@ function initializeImportsAndExports(
 
 export const __initializeImportsAndExports: any = initializeImportsAndExports; // don't want to export the type
 
+// the methods would be visible to EMCC linker
+// --- keep in sync with dotnet.cjs.lib.js ---
+const mono_wasm_threads_exports = !MonoWasmThreads ? undefined : {
+    // mono-threads-wasm.c
+    mono_wasm_pthread_on_pthread_attached,
+};
+
 // the methods would be visible to EMCC linker
 // --- keep in sync with dotnet.cjs.lib.js ---
 export const __linker_exports: any = {
@@ -376,7 +405,10 @@ export const __linker_exports: any = {
     // pal_crypto_webworker.c
     dotnet_browser_can_use_subtle_crypto_impl,
     dotnet_browser_simple_digest_hash,
-    dotnet_browser_sign
+    dotnet_browser_sign,
+
+    // threading exports, if threading is enabled
+    ...mono_wasm_threads_exports,
 };
 
 const INTERNAL: any = {
@@ -450,4 +482,4 @@ class RuntimeList {
 
 export function get_dotnet_instance(): DotnetPublicAPI {
     return exportedAPI;
-}
+}

src/mono/wasm/runtime/imports.ts

@@ -21,6 +21,7 @@ export let ENVIRONMENT_IS_NODE: boolean;
 export let ENVIRONMENT_IS_SHELL: boolean;
 export let ENVIRONMENT_IS_WEB: boolean;
 export let ENVIRONMENT_IS_WORKER: boolean;
+export let ENVIRONMENT_IS_PTHREAD: boolean;
 export let locateFile: Function;
 export let quit: Function;
 export let ExitStatus: ExitStatusError;
@@ -33,7 +34,7 @@ export interface ExitStatusError {
 
 // eslint-disable-next-line @typescript-eslint/explicit-module-boundary-types
 export function setImportsAndExports(
-    imports: { isESM: boolean, isNode: boolean, isShell: boolean, isWeb: boolean, isWorker: boolean, locateFile: Function, ExitStatus: ExitStatusError, quit_: Function, requirePromise: Promise<Function> },
+    imports: { isESM: boolean, isNode: boolean, isShell: boolean, isWeb: boolean, isWorker: boolean, isPThread: boolean, locateFile: Function, ExitStatus: ExitStatusError, quit_: Function, requirePromise: Promise<Function> },
     exports: { mono: any, binding: any, internal: any, module: any, marshaled_exports: any, marshaled_imports: any },
 ): void {
     MONO = exports.mono;
@@ -49,6 +50,7 @@ export function setImportsAndExports(
     ENVIRONMENT_IS_SHELL = imports.isShell;
     ENVIRONMENT_IS_WEB = imports.isWeb;
     ENVIRONMENT_IS_WORKER = imports.isWorker;
+    ENVIRONMENT_IS_PTHREAD = imports.isPThread;
     locateFile = imports.locateFile;
     quit = imports.quit_;
     ExitStatus = imports.ExitStatus;