Assert failure(PID 2292 [0x000008f4], Thread: 2312 [0x0908]): !g_pConfig->fAssertOnFailFast() || (m_Next == FRAME_TOP) || (*m_Next->GetGSCookiePtr() == GetProcessGSCookie())

[AriNuer]

Job:
runtime-coreclr gcstress0x3-gcstress0xc:20200315.1
Error message:

Assert failure(PID 2292 [0x000008f4], Thread: 2312 [0x0908]): !g_pConfig->fAssertOnFailFast() || (m_Next == FRAME_TOP) || (*m_Next->GetGSCookiePtr() == GetProcessGSCookie())

CORECLR! Frame::Push + 0x1A9 (0x00007ff956303ad9)
CORECLR! Frame::Push + 0xDE (0x00007ff956303c0a)
CORECLR! Thread::RedirectedHandledJITCase + 0x1AD (0x00007ff9561b7a05)
CORECLR! RedirectedHandledJITCaseForGCStress_Stub + 0x26 (0x00007ff956615a06)
<no module>! <no symbol> + 0x0 (0x00007ff8f72c00f5)
<no module>! <no symbol> + 0x0 (0x00007ff8f6e0f518)
<no module>! <no symbol> + 0x0 (0x0000000000003039)
<no module>! <no symbol> + 0x0 (0x000000000000018c)
<no module>! <no symbol> + 0x0 (0xffffffffffffffff)
<no module>! <no symbol> + 0x0 (0x9abcdef012345678)
File: F:\workspace\_work\1\s\src\coreclr\src\vm\frames.cpp Line: 417
Image: C:\h\w\B366092E\p\CoreRun.exe

Return code: 1\r\nRaw output file: C:\h\w\B366092E\w\AA26098B\e\Interop\NativeCallable\Reports\Interop.NativeCallable\NativeCallableTest\NativeCallableTest.output.txt
Raw output:
BEGIN EXECUTION
"C:\h\w\B366092E\p\corerun.exe" NativeCallableTest.dll
Running TestNativeCallableValid...
Running TestNativeCallableValid_OnNewNativeThread...
Running TestNativeCallableValid_PrepareMethod...
Running NegativeTest_NonStaticMethod...
Running NegativeTest_ViaDelegate...
Running NegativeTest_NonBlittable...
Running NegativeTest_NonInstantiatedGenericArguments...
Running NegativeTest_InstantiatedGenericArguments...
Running NegativeTest_FromInstantiatedGenericClass...
Running TestNativeCallableViaUnmanagedCalli...
Running TestNativeCallableValid_ThrowException...
Expected: 100
Actual: -1073740286
END EXECUTION - FAILED
FAILED
Test Harness Exitcode is : 1
To run the test:
> set CORE_ROOT=C:\h\w\B366092E\p
> C:\h\w\B366092E\w\AA26098B\e\Interop\NativeCallable\NativeCallableTest\NativeCallableTest.cmd
Expected: True
Actual: False

Stack trace:
at Interop_NativeCallable._NativeCallableTest_NativeCallableTest_._NativeCallableTest_NativeCallableTest_cmd() in F:\workspace\_work\1\s\artifacts\tests\coreclr\Windows_NT.x64.Checked\TestWrappers\Interop.NativeCallable\Interop.NativeCallable.XUnitWrapper.cs:line 140
Details:
https://dev.azure.com/dnceng/public/_build/results?buildId=560162&view=ms.vss-test-web.build-test-results-tab&runId=17645012&resultId=101118&paneView=debug

[AaronRobinsonMSFT]

@jkotas or @janvorli Any suggestions here on how to address this for Windows would be helpful. I have a fix for #33612, but this one seems related to the exception handling side of things that I am less knowledgable about how to mitigate.

[AaronRobinsonMSFT]

This is easy to reproduce locally. Looks like an issue with TestNativeCallableValid_ThrowException() test.

runtime/src/coreclr/tests/src/Interop/NativeCallable/NativeCallableTest.cs

Lines 210 to 255 in a1af0f2

	private const int CallbackThrowsErrorCode = 27;
	[NativeCallable]
	public static int CallbackThrows(int val)
	{
	throw new Exception() { HResult = CallbackThrowsErrorCode };
	}
	public static void TestNativeCallableValid_ThrowException()
	{
	Console.WriteLine($"Running {nameof(TestNativeCallableValid_ThrowException)}...");
	/*
	void NativeCallableValid_ThrowException()
	{
	.locals init ([0] native int ptr)
	IL_0000: nop
	IL_0001: ldftn int32 CallbackThrows(int32)
	IL_0007: stloc.0
	IL_0008: ldloc.0
	IL_0009: ldc.i4 <n> local
	IL_000e: call bool NativeCallableDll::CallManagedProcCatchException(native int, int)
	IL_0013: ret
	}
	*/
	DynamicMethod testNativeCallable = new DynamicMethod("NativeCallableValid_ThrowException", typeof(int), null, typeof(Program).Module);
	ILGenerator il = testNativeCallable.GetILGenerator();
	il.DeclareLocal(typeof(IntPtr));
	il.Emit(OpCodes.Nop);
	// Get native function pointer of the callback
	il.Emit(OpCodes.Ldftn, typeof(Program).GetMethod(nameof(CallbackThrows)));
	il.Emit(OpCodes.Stloc_0);
	il.Emit(OpCodes.Ldloc_0);
	int n = 12345;
	il.Emit(OpCodes.Ldc_I4, n);
	il.Emit(OpCodes.Call, typeof(NativeCallableDll).GetMethod("CallManagedProcCatchException"));
	il.Emit(OpCodes.Ret);
	var testNativeMethod = (IntNativeMethodInvoker)testNativeCallable.CreateDelegate(typeof(IntNativeMethodInvoker));
	// Method should have thrown and caught an exception.
	Assert.AreEqual(-1, testNativeMethod());
	}

[AaronRobinsonMSFT]

Bah. I think this is my bad. @jkotas If I recall a conversation about exceptions and the NativeCallableAttribute, the expectation here is that they don't mix at all and we shouldn't even be trying to validate them right? If so I will disable those tests since I added them just to have a set of scenarios I could validate fail. Looks like on GCStress they really don't work due to the stack walking logic in

runtime/src/coreclr/src/vm/exceptionhandling.cpp

Line 6392 in 4ceccbb

bool ExceptionTracker::HasFrameBeenUnwoundByAnyActiveException(CrawlFrame * pCF)

[jkotas]

On Windows, the managed exception should get converted to SEH exception on the NativeCallable boundary.
On Unix, the managed exception should become unhanded on the NativeCallable boundary.

[jkotas]

I think we are missing "unwind and continue" on the NativeCallable boundary. @janvorli Do you have thoughts about the best way to fix that?