Add async implementation of EventLoopGroup.shutdownGracefully to _NIOConcurrency

[simonjbeaumont]

Adds a new extension to EventLoopGroup in _NIOConcurrency to provide an asynchronous shutdownGracefully().

Motivation

Shutting down an event loop was special because it used a completion handler and executed on a DispatchQueue because there would be no event loop to use an EventLoopFuture.

Now we have the new Swift concurrency features, we can provide an async shutdownGracefully.

Fixes #1878.

Modifications

Adds EventLoopGroup.shutdownGracefully() async and some code to use it in NIOAsyncAwaitDemo.

There's no obvious way to ask an ELG if it has been shutdown but I made the following local-only changes to verify that this performs as expected:

diff --git a/Sources/NIO/EventLoop.swift b/Sources/NIO/EventLoop.swift
index d18a33dd..972b3b05 100644
--- a/Sources/NIO/EventLoop.swift
+++ b/Sources/NIO/EventLoop.swift
@@ -840 +840 @@ public final class MultiThreadedEventLoopGroup: EventLoopGroup {
-    private enum RunState {
+    public enum RunState {
@@ -852 +852 @@ public final class MultiThreadedEventLoopGroup: EventLoopGroup {
-    private var runState: RunState = .running
+    public var runState: RunState = .running
diff --git a/Sources/NIOAsyncAwaitDemo/main.swift b/Sources/NIOAsyncAwaitDemo/main.swift
index 97a9c0c3..f8b29ace 100644
--- a/Sources/NIOAsyncAwaitDemo/main.swift
+++ b/Sources/NIOAsyncAwaitDemo/main.swift
@@ -60,0 +61 @@ func main() async {
+        print("elg state: \(group.runState)")  // .running
@@ -61,0 +63,7 @@ func main() async {
+        print("elg state: \(group.runState)")  // .closed(nil)
+
+        /// This causes the following error:
+        /// ERROR: Cannot schedule tasks on an EventLoop that has already shut down. This will be upgraded to a forced crash in future SwiftNIO versions.
+        group.next().execute {
+            print("This will not execute.")
+        }

[swift-server-bot]

Can one of the admins verify this patch?

[swift-server-bot]

Can one of the admins verify this patch?

[swift-server-bot]

Can one of the admins verify this patch?

[swift-server-bot]

Can one of the admins verify this patch?

[swift-server-bot]

Can one of the admins verify this patch?

[swift-server-bot]

Can one of the admins verify this patch?

[swift-server-bot]

Can one of the admins verify this patch?

[swift-server-bot]

Can one of the admins verify this patch?

[swift-server-bot]

Can one of the admins verify this patch?

[swift-server-bot]

Can one of the admins verify this patch?

[swift-server-bot]

Can one of the admins verify this patch?

[swift-server-bot]

Can one of the admins verify this patch?

[swift-server-bot]

Can one of the admins verify this patch?

[swift-server-bot]

Can one of the admins verify this patch?

[Lukasa]

Thanks, looks good! Just two small notes in the diff.

[Lukasa]

@swift-nio-bot add to allowlist