Port the Shared Mutex logic from CoreCLR PAL to work alongside the managed wait subsystem

[jkoritzinsky]

Ported the implementation of cross-process shared memory from src/coreclr/pal/src/sharedmemory/sharedmemory.cpp and the cross-platform mutex (built on top of that) from src/coreclr/pal/src/syncobj/mutex.cpp to managed code, using the same syscalls, just through the libraries PAL layer.

Set up NativeAOT and Mono to use this implementation when targeting a non-mobile-like platform and use the existing (in-process) implementation for mobile-like platforms.

This change allows us to move CoreCLR to use the managed wait subsystem + Named Mutexes on Unix instead of the existing unmanaged wait subsystem used by both the runtime and managed code. That will further allow us to slowly chip away at the unmanaged wait subsystem and move usages to use APIs defined in the minipal with less overhead.

I've attempted to match the existing behavior per platform.

On Linux, illumos, tizen, etc, we will use a robust pthread mutex as the backing implementation for the cross-process mutex.

On FreeBSD, we will use a manual mutex implementation because FreeBSD doesn't support truly robust pthread mutexes.

On macOS, we will use a manual mutex implementation as only Apple Silicon supports them (no Rosetta support), and we want to support using named mutexes across processes even if one is Rosetta and another isn't.

Fixes #48720

[Copilot]

Pull Request Overview

This PR ports the cross-process shared mutex logic from the CoreCLR PAL into managed code using the native PAL layer, enabling named mutex support within the managed wait subsystem on Unix platforms (excluding mobile-like targets). It extends both NativeAOT and Mono to use this implementation, removes previous test exclusions, and wires up the necessary native exports and interop signatures.

• Introduces LowLevelCrossPlatformMutex PAL APIs and implements them in pal_threading.c for Unix.
• Adds stubs in WASI (pal_threading_wasi.c), new interop methods, and build definition entries (FeatureCrossProcessMutex).
• Updates managed code to use IWaitableObject abstraction, implements NamedMutex.Unix.cs, and adjusts Mutex.Unix.cs and tests.

Reviewed Changes

Copilot reviewed 19 out of 19 changed files in this pull request and generated 1 comment.

Show a summary per file

File	Description
src/native/libs/System.Native/pal_threading_wasi.c	Add stubbed LowLevelCrossPlatformMutex for WASI
src/native/libs/System.Native/pal_threading.h	Declare new PThread and CrossPlatformMutex PAL APIs
src/native/libs/System.Native/pal_threading.c	Implement LowLevelCrossPlatformMutex on Unix platforms
src/native/libs/System.Native/pal_process.{h,c}	Add SystemNative_GetPageSize native API
src/native/libs/System.Native/entrypoints.c	Export GetPageSize and cross-platform mutex methods
src/libraries/System.Threading/tests/MutexTests.cs	Remove exclusions for Mono and NativeAOT in named mutex tests
src/libraries/System.Private.CoreLib/src/System/Threading/WaitSubsystem*.Unix.cs files	Introduce IWaitableObject, integrate named mutex logic
src/libraries/System.Private.CoreLib/src/System/Threading/NamedMutex.Unix.cs	Implement managed cross-process named mutex operations
src/libraries/System.Private.CoreLib/src/System/Threading/Mutex.Unix.cs	Pass user-scope flag through creation/open methods
src/libraries/System.Private.CoreLib/src/System/IO/SharedMemoryManager.Unix.cs	New shared-memory-backed processing for named mutex data
src/libraries/Common/src/Interop/Unix/System.Native/Interop.LowLevelCrossProcessMutex.cs	Add P/Invoke signatures for new PAL mutex APIs
src/libraries/Common/src/Interop/Unix/System.Native/Interop.GetPageSize.cs	Add P/Invoke signature for page size API
Various .csproj and .projitems files	Define FEATURE_CROSS_PROCESS_MUTEX and include new sources

Comments suppressed due to low confidence (2)

src/native/libs/System.Native/pal_threading_wasi.c:99

• The struct is named LowLevelCrossProcessMutex but all related functions use LowLevelCrossPlatformMutex in their names, causing an inconsistency that could confuse readers or lead to linkage issues. Consider renaming the struct to match (LowLevelCrossPlatformMutex) or vice versa.

struct LowLevelCrossProcessMutex

src/native/libs/System.Native/pal_threading.h:37

• The header declares SystemNative_PThreadMutex_* methods but the WASI implementation (pal_threading_wasi.c) does not provide definitions for these functions, which will lead to linker errors. You should either stub out these methods in the WASI file or conditionally exclude their declarations on WASI.

PALEXPORT int32_t SystemNative_PThreadMutex_Init(void* mutex);

[jkotas]

This should wait for .NET 11.

[risc-vv]

@dotnet/samsung Could you please take a look? These changes may be related to riscv64.

[jkotas]

LGTM otherwise. Thank you!

[jkotas]

Thanks!

[jkoritzinsky]

/ba-g timeouts