Merge dotnet/main into feature/HMF-removal/JIT_Box

Author: am11

docs/design/specs/runtime-async.md

@@ -18,14 +18,14 @@ Applicability of `MethodImplOptions.Async`:
 * The `[MethodImpl(MethodImplOptions.Async)]` only has effect when applied to method definitions with CIL implementation.
 * Async method definitions are only valid inside async-capable assemblies. An async-capable assembly is one which references a corlib containing an `abstract sealed class RuntimeFeature` with a `public const string` field member named `Async`.
 * Combining `MethodImplOptions.Async` with `MethodImplOptions.Synchronized` is invalid.
-* Applying `MethodImplOptions.Async` to methods with `byref` or `ref-like` parameters is invalid.
+* Applying `MethodImplOptions.Async` to methods with a `byref` or `ref-like` return value is invalid.
 * Applying `MethodImplOptions.Async` to vararg methods is invalid.
 
 _[Note: these rules operate before generic substitution, meaning that a method which only meets requirements after substitution would not be considered as valid.]_
 
 Sync methods are all other methods.
 
-Unlike sync methods, async methods support suspension. Suspension allows async methods to yield control flow back to their caller at certain well-defined suspension points, and resume execution of the remaining method at a later time or location, potentially on another thread.
+Unlike sync methods, async methods support suspension. Suspension allows async methods to yield control flow back to their caller at certain well-defined suspension points, and resume execution of the remaining method at a later time or location, potentially on another thread. Suspension points are where suspension may occur, but suspension is not required if all Task-like objects are completed.
 
 Async methods also do not have matching return type conventions as sync methods. For sync methods, the stack should contain a value convertible to the stated return type before the `ret` instruction. For async methods, the stack should be empty in the case of `Task` or `ValueTask`, or the type argument in the case of `Task<T>` or `ValueTask<T>`.
 
@@ -64,13 +64,12 @@ Async methods support suspension using one of the following methods:
 
 These methods are only legal to call inside async methods. The `...AwaitAwaiter...` methods will have semantics analogous to the current `AsyncTaskMethodBuilder.AwaitOnCompleted/AwaitUnsafeOnCompleted` methods. After calling either method, it can be presumed that the task or awaiter has completed. The `Await` methods perform suspension like the `AwaitAwaiter...` methods, but are optimized for calling on the return value of a call to an async method. To achieve maximum performance, the IL sequence of two `call` instructions -- one to the async method and immediately one to the `Await` method -- should be preferred.
 
-Only local variables which are "hoisted" may be used across suspension points. That is, only "hoisted" local variables will have their state preserved after returning from a suspension. On methods with the `localsinit` flag set, non-"hoisted" local variables will be initialized to their default value when resuming from suspension. Otherwise, these variables will have an undefined value. To identify "hoisted" local variables, they must have an optional custom modifier to the `System.Runtime.CompilerServices.HoistedLocal` class, which will be a new .NET runtime API. This custom modifier must be the last custom modifier on the variable. It is invalid for by-ref variables, or variables with a by-ref-like type, to be marked hoisted. Hoisted local variables are stored in managed memory and cannot be converted to unmanaged pointers without explicit pinning.
-The code generator is free to ignore the `HoistedLocal` modifier if it can prove that this makes no observable difference in the execution of the generated program. This can be observable in diagnostics since it may mean the value of a local with the `HoistedLocal` modifier will not be available after certain suspension points.
+Local variables used across suspension points are considered "hoisted." That is, only "hoisted" local variables will have their state preserved after returning from a suspension. By-ref variables may not be hoisted across suspension points, and any read of a by-ref variable after a suspension point will produce null. Structs containing by-ref variables will also not be hoisted across suspension points and will have their default value after a suspension point. 
+In the same way, pinning locals may not be "hoisted" across suspension points and will have `null` value after a suspension point.
 
 Async methods have some temporary restrictions with may be lifted later:
 * The `tail` prefix is forbidden
 * Usage of the `localloc` instruction is forbidden
-* Pinning locals may not be marked `HoistedLocal`
 
 Other restrictions are likely to be permanent, including
 * By-ref locals cannot be hoisted across suspension points

docs/workflow/building/coreclr/ios.md

@@ -1,4 +1,4 @@
-# Cross Compilation for iOS Simulator on macOS
+# Cross Compilation for iOS/tvOS Simulator on macOS
 
 ## Requirements
 
@@ -9,7 +9,7 @@ Build requirements are the same as for building native CoreCLR on macOS. iPhone
 Build the runtime pack and tools with
 
 ```
-./build.sh clr+clr.runtime+libs+packs -os [iossimulator/maccatalyst] -arch [x64/arm64] -cross -c Release
+./build.sh clr+clr.runtime+libs+packs -os [iossimulator/tvossimulator/maccatalyst] -arch [x64/arm64] -cross -c Release
 ```
 
 ## Running the sample iOS app

src/coreclr/CMakeLists.txt

@@ -105,9 +105,8 @@ if(CLR_CMAKE_HOST_UNIX)
       add_linker_flag(-Wl,-z,notext)
     endif()
 
-    if(NOT CLR_CMAKE_HOST_TVOS)
-      add_subdirectory(pal)
-    endif()
+    add_subdirectory(pal)
+
     if(NOT CLR_CMAKE_HOST_MACCATALYST AND NOT CLR_CMAKE_HOST_IOS AND NOT CLR_CMAKE_HOST_TVOS)
       add_subdirectory(hosts)
     endif()
@@ -135,9 +134,7 @@ add_subdirectory(${CLR_SRC_NATIVE_DIR}/containers containers)
 add_subdirectory(${CLR_SRC_NATIVE_DIR}/eventpipe eventpipe)
 add_subdirectory(${CLR_SRC_NATIVE_DIR}/minipal shared_minipal)
 
-if(NOT CLR_CMAKE_HOST_TVOS)
-  add_subdirectory(debug/debug-pal)
-endif()
+add_subdirectory(debug/debug-pal)
 
 add_subdirectory(minipal)
 
@@ -273,36 +270,34 @@ if(CLR_CMAKE_HOST_UNIX)
   add_subdirectory(nativeresources)
 endif(CLR_CMAKE_HOST_UNIX)
 
-if(NOT CLR_CMAKE_HOST_TVOS)
-  add_subdirectory(utilcode)
-  add_subdirectory(inc)
+add_subdirectory(utilcode)
+add_subdirectory(inc)
 
-  if (CLR_CMAKE_BUILD_TOOLS)
-    add_subdirectory(ilasm)
-    add_subdirectory(ildasm)
-  endif(CLR_CMAKE_BUILD_TOOLS)
-  add_subdirectory(gcinfo)
+if (CLR_CMAKE_BUILD_TOOLS)
+  add_subdirectory(ilasm)
+  add_subdirectory(ildasm)
+endif(CLR_CMAKE_BUILD_TOOLS)
+add_subdirectory(gcinfo)
 
-  if (NOT CLR_CMAKE_TARGET_ARCH_WASM)
-    add_subdirectory(jit)
-  endif()
+if (NOT CLR_CMAKE_TARGET_ARCH_WASM)
+  add_subdirectory(jit)
+endif()
 
-  add_subdirectory(vm)
-  add_subdirectory(md)
+add_subdirectory(vm)
+add_subdirectory(md)
 
-  if (NOT CLR_CMAKE_TARGET_ARCH_WASM)
-    add_subdirectory(debug)
-  endif()
+if (NOT CLR_CMAKE_TARGET_ARCH_WASM)
+  add_subdirectory(debug)
+endif()
 
-  add_subdirectory(binder)
-  add_subdirectory(classlibnative)
-  add_subdirectory(dlls)
+add_subdirectory(binder)
+add_subdirectory(classlibnative)
+add_subdirectory(dlls)
 
-  if (NOT CLR_CMAKE_TARGET_ARCH_WASM)
-    add_subdirectory(unwinder)
-    add_subdirectory(interop)
-  endif()
-endif(NOT CLR_CMAKE_HOST_TVOS)
+if (NOT CLR_CMAKE_TARGET_ARCH_WASM)
+  add_subdirectory(unwinder)
+  add_subdirectory(interop)
+endif()
 
 if(NOT CLR_CMAKE_HOST_MACCATALYST AND NOT CLR_CMAKE_HOST_IOS AND NOT CLR_CMAKE_HOST_TVOS)
   add_subdirectory(tools)

src/coreclr/dlls/mscoree/coreclr/CMakeLists.txt

@@ -228,7 +228,7 @@ endif(CLR_CMAKE_TARGET_WIN32)
 
 # add the install targets
 install_clr(TARGETS coreclr DESTINATIONS . sharedFramework COMPONENT runtime)
-if(CLR_CMAKE_HOST_MACCATALYST OR CLR_CMAKE_HOST_IOS)
+if(CLR_CMAKE_HOST_MACCATALYST OR CLR_CMAKE_HOST_IOS OR CLR_CMAKE_HOST_TVOS)
     install_clr(TARGETS coreclr_static DESTINATIONS . sharedFramework COMPONENT runtime)
 endif()
 

src/coreclr/pal/src/config.h.in

@@ -45,6 +45,7 @@
 #cmakedefine01 HAVE__LWP_SELF
 #cmakedefine01 HAVE_MACH_THREADS
 #cmakedefine01 HAVE_MACH_EXCEPTIONS
+#cmakedefine01 HAVE_SIGALTSTACK
 #cmakedefine01 HAVE_VM_ALLOCATE
 #cmakedefine01 HAVE_VM_READ
 #cmakedefine01 HAVE_DIRECTIO

src/coreclr/pal/src/configure.cmake

@@ -120,7 +120,20 @@ check_function_exists(statvfs HAVE_STATVFS)
 check_function_exists(thread_self HAVE_THREAD_SELF)
 check_function_exists(_lwp_self HAVE__LWP_SELF)
 check_function_exists(pthread_mach_thread_np HAVE_MACH_THREADS)
-check_function_exists(thread_set_exception_ports HAVE_MACH_EXCEPTIONS)
+check_cxx_source_compiles("
+#include <mach/mach.h>
+int main(int argc, char **argv) {
+  static mach_port_name_t port;
+  thread_set_exception_ports(mach_thread_self(), EXC_MASK_BAD_ACCESS, port, EXCEPTION_DEFAULT, MACHINE_THREAD_STATE);
+  return 0;
+}" HAVE_MACH_EXCEPTIONS)
+check_cxx_source_compiles("
+#include <signal.h>
+#include <stdlib.h>
+int main(int argc, char **argv) {
+  sigaltstack(NULL, NULL);
+  return 0;
+}" HAVE_SIGALTSTACK)
 check_function_exists(vm_allocate HAVE_VM_ALLOCATE)
 check_function_exists(vm_read HAVE_VM_READ)
 check_function_exists(directio HAVE_DIRECTIO)

src/coreclr/pal/src/debug/debug.cpp

@@ -88,7 +88,7 @@ const BOOL DBG_DETACH       = FALSE;
 #endif
 static const char PAL_OUTPUTDEBUGSTRING[]    = "PAL_OUTPUTDEBUGSTRING";
 
-#ifdef _DEBUG
+#if defined(_DEBUG) && !defined(TARGET_IOS) && !defined(TARGET_TVOS)
 #define ENABLE_RUN_ON_DEBUG_BREAK 1
 #endif // _DEBUG
 

src/coreclr/pal/src/exception/seh.cpp

@@ -304,9 +304,9 @@ PAL_ERROR SEHEnable(CPalThread *pthrCurrent)
 {
 #if HAVE_MACH_EXCEPTIONS
     return pthrCurrent->EnableMachExceptions();
-#elif defined(__linux__) || defined(__FreeBSD__) || defined(__NetBSD__) || defined(__sun) || defined(__HAIKU__) || defined(__wasm__)
+#elif defined(__linux__) || defined(__FreeBSD__) || defined(__NetBSD__) || defined(__sun) || defined(__HAIKU__) || defined(__APPLE__) || defined(__wasm__)
     return NO_ERROR;
-#else// HAVE_MACH_EXCEPTIONS
+#else // HAVE_MACH_EXCEPTIONS
 #error not yet implemented
 #endif // HAVE_MACH_EXCEPTIONS
 }
@@ -329,7 +329,7 @@ PAL_ERROR SEHDisable(CPalThread *pthrCurrent)
 {
 #if HAVE_MACH_EXCEPTIONS
     return pthrCurrent->DisableMachExceptions();
-#elif defined(__linux__) || defined(__FreeBSD__) || defined(__NetBSD__) || defined(__sun) || defined(__HAIKU__) || defined(__wasm__)
+#elif defined(__linux__) || defined(__FreeBSD__) || defined(__NetBSD__) || defined(__sun) || defined(__HAIKU__) || defined(__APPLE__) || defined(__wasm__)
     return NO_ERROR;
 #else // HAVE_MACH_EXCEPTIONS
 #error not yet implemented

src/coreclr/pal/src/exception/signal.cpp

@@ -188,7 +188,7 @@ BOOL SEHInitializeSignals(CorUnix::CPalThread *pthrCurrent, DWORD flags)
         handle_signal(SIGINT, sigint_handler, &g_previous_sigint, 0 /* additionalFlags */, true /* skipIgnored */);
         handle_signal(SIGQUIT, sigquit_handler, &g_previous_sigquit, 0 /* additionalFlags */, true /* skipIgnored */);
 
-#if HAVE_MACH_EXCEPTIONS
+#if HAVE_MACH_EXCEPTIONS || !HAVE_SIGALTSTACK
         handle_signal(SIGSEGV, sigsegv_handler, &g_previous_sigsegv);
 #else
         handle_signal(SIGTRAP, sigtrap_handler, &g_previous_sigtrap);
@@ -569,6 +569,7 @@ extern "C" void signal_handler_worker(int code, siginfo_t *siginfo, void *contex
     RtlRestoreContext(&returnPoint->context, NULL);
 }
 
+#if HAVE_SIGALTSTACK
 /*++
 Function :
     SwitchStackAndExecuteHandler
@@ -609,6 +610,7 @@ static bool SwitchStackAndExecuteHandler(int code, siginfo_t *siginfo, void *con
 
     return pReturnPoint->returnFromHandler;
 }
+#endif
 
 #endif // !HAVE_MACH_EXCEPTIONS
 
@@ -644,6 +646,10 @@ static void sigsegv_handler(int code, siginfo_t *siginfo, void *context)
         {
             if (GetCurrentPalThread())
             {
+#if defined(TARGET_TVOS)
+                (void)!write(STDERR_FILENO, StackOverflowMessage, sizeof(StackOverflowMessage) - 1);
+                PROCAbort(SIGSEGV, siginfo);
+#else // TARGET_TVOS
                 size_t handlerStackTop = __sync_val_compare_and_swap((size_t*)&g_stackOverflowHandlerStack, (size_t)g_stackOverflowHandlerStack, 0);
                 if (handlerStackTop == 0)
                 {
@@ -672,6 +678,7 @@ static void sigsegv_handler(int code, siginfo_t *siginfo, void *context)
                     PROCAbort(SIGSEGV, siginfo);
                 }
                 (void)!write(STDERR_FILENO, StackOverflowHandlerReturnedMessage, sizeof(StackOverflowHandlerReturnedMessage) - 1);
+#endif // TARGET_TVOS
             }
             else
             {
@@ -686,6 +693,7 @@ static void sigsegv_handler(int code, siginfo_t *siginfo, void *context)
             // Now that we know the SIGSEGV didn't happen due to a stack overflow, execute the common
             // hardware signal handler on the original stack.
 
+#if HAVE_SIGALTSTACK
             if (GetCurrentPalThread() && IsRunningOnAlternateStack(context))
             {
                 if (SwitchStackAndExecuteHandler(code, siginfo, context, 0 /* sp */)) // sp == 0 indicates execution on the original stack
@@ -694,6 +702,7 @@ static void sigsegv_handler(int code, siginfo_t *siginfo, void *context)
                 }
             }
             else
+#endif
             {
                 // The code flow gets here when the signal handler is not running on an alternate stack or when it wasn't created
                 // by coreclr. In both cases, we execute the common_signal_handler directly.

src/coreclr/pal/src/include/pal/thread.hpp

@@ -572,7 +572,7 @@ namespace CorUnix
             m_pNext = pNext;
         };
 
-#if !HAVE_MACH_EXCEPTIONS
+#if !HAVE_MACH_EXCEPTIONS && HAVE_SIGALTSTACK
         BOOL
         EnsureSignalAlternateStack(
             void

src/coreclr/pal/src/init/sxs.cpp

@@ -64,7 +64,7 @@ AllocatePalThread(CPalThread **ppThread)
         goto exit;
     }
 
-#if !HAVE_MACH_EXCEPTIONS
+#if !HAVE_MACH_EXCEPTIONS && HAVE_SIGALTSTACK
     // Ensure alternate stack for SIGSEGV handling. Our SIGSEGV handler is set to
     // run on an alternate stack and the stack needs to be allocated per thread.
     if (!pThread->EnsureSignalAlternateStack())

src/coreclr/pal/src/thread/process.cpp

@@ -513,6 +513,9 @@ CorUnix::InternalCreateProcess(
     LPPROCESS_INFORMATION lpProcessInformation
     )
 {
+#ifdef TARGET_TVOS
+    return ERROR_NOT_SUPPORTED;
+#else
     PAL_ERROR palError = NO_ERROR;
     IPalObject *pobjProcess = NULL;
     IPalObject *pobjProcessRegistered = NULL;
@@ -1049,6 +1052,7 @@ CorUnix::InternalCreateProcess(
     }
 
     return palError;
+#endif // !TARGET_TVOS
 }
 
 
@@ -2180,7 +2184,7 @@ PROCCreateCrashDump(
     INT cbErrorMessageBuffer,
     bool serialize)
 {
-#if defined(TARGET_IOS)
+#if defined(TARGET_IOS) || defined(TARGET_TVOS)
     return FALSE;
 #else
     _ASSERTE(argv.size() > 0);
@@ -2310,7 +2314,7 @@ PROCCreateCrashDump(
         }
     }
     return true;
-#endif // !TARGET_IOS
+#endif // !TARGET_IOS && !TARGET_TVOS
 }
 
 /*++

src/coreclr/pal/src/thread/thread.cpp

@@ -134,7 +134,7 @@ static void InternalEndCurrentThreadWrapper(void *arg)
        will lock its own critical section */
     LOADCallDllMain(DLL_THREAD_DETACH, NULL);
 
-#if !HAVE_MACH_EXCEPTIONS
+#if !HAVE_MACH_EXCEPTIONS && HAVE_SIGALTSTACK
     pThread->FreeSignalAlternateStack();
 #endif // !HAVE_MACH_EXCEPTIONS
 
@@ -1552,7 +1552,7 @@ CPalThread::ThreadEntry(
     }
 #endif // HAVE_SCHED_GETAFFINITY && HAVE_SCHED_SETAFFINITY
 
-#if !HAVE_MACH_EXCEPTIONS
+#if !HAVE_MACH_EXCEPTIONS && HAVE_SIGALTSTACK
     if (!pThread->EnsureSignalAlternateStack())
     {
         ASSERT("Cannot allocate alternate stack for SIGSEGV!\n");
@@ -2275,7 +2275,7 @@ CPalThread::WaitForStartStatus(
     return m_fStartStatus;
 }
 
-#if !HAVE_MACH_EXCEPTIONS
+#if !HAVE_MACH_EXCEPTIONS && HAVE_SIGALTSTACK
 /*++
 Function :
     EnsureSignalAlternateStack

src/libraries/Common/src/Interop/Unix/System.Security.Cryptography.Native/Interop.EvpPkey.cs

@@ -47,6 +47,9 @@ internal static int UpRefEvpPkey(SafeEvpPKeyHandle handle)
         [LibraryImport(Libraries.CryptoNative, EntryPoint = "CryptoNative_EvpPKeyType")]
         internal static partial EvpAlgorithmId EvpPKeyType(SafeEvpPKeyHandle handle);
 
+        [LibraryImport(Libraries.CryptoNative, EntryPoint = "CryptoNative_EvpPKeyFamily")]
+        internal static partial EvpAlgorithmFamilyId EvpPKeyFamily(SafeEvpPKeyHandle handle);
+
         [LibraryImport(Libraries.CryptoNative)]
         private static unsafe partial SafeEvpPKeyHandle CryptoNative_DecodeSubjectPublicKeyInfo(
             byte* buf,
@@ -327,5 +330,14 @@ internal enum EvpAlgorithmId
             DSA = 116,
             ECC = 408,
         }
+
+        internal enum EvpAlgorithmFamilyId
+        {
+            Unknown = 0,
+            RSA = 1,
+            DSA = 2,
+            ECC = 3,
+            MLKem = 4,
+        }
     }
 }