Skip to content

EventPipe shutdown race condition in .net7/.net6 #89621

@lateralusX

Description

@lateralusX

Description

The fix for f39e05a added a condition to close down the listener's IPC channel during shutdown. Turns out that this can cause a race with the diagnostic server thread if its in the process of using that handle as part of a GetOverlappedResult call that in turn will end up closing the handle:

CloseHandle (ipc->pipe);

This probably will go unnoticed on release builds, but on debug build it could trigger asserts due to internal heap corruption or attempt to close and invalid handle if the race gets exposed.

Currently EventPipe doesn't have any coordination mechanism during shutdown to make sure diagnostic server thread stops during shutdown, so it can't close any of the resources since they might still be used.

Fix is to revert

if (is_shutdown && ipc->mode != DS_IPC_CONNECTION_MODE_LISTEN) {
to keep ipc->pipe around during shutdown:

@@ -634,13 +634,11 @@ ds_ipc_close (
	ds_ipc_error_callback_func callback)
{
	EP_ASSERT (ipc != NULL);

	// don't attempt cleanup on shutdown and let the OS handle it
-    // except in the case of listen pipes - if they leak the process
-    // will fail to reinitialize the pipe for embedding scenarios.
-	if (is_shutdown && ipc->mode != DS_IPC_CONNECTION_MODE_LISTEN) {
+	if (is_shutdown) {
		if (callback)
			callback ("Closing without cleaning underlying handles", 100);
		return;
	}

Reproduction Steps

Running EventPipe test like src\tests\tracing\eventpipe\rundownvalidation using debug build running under debugger.

Expected behavior

No race.

Actual behavior

If race is hit, process will crash.

Regression?

Yes

Known Workarounds

No response

Configuration

No response

Other information

No response

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions