Auto-tuning DotNetty batching + removing scheduler from batching system

[Aaronontheweb]

Ported the FlushConsolidationHandler from Netty and used it to auto-tune batching inside Akka.Remote without having to set explicit thresholds and without having to rely on the IScheduler.

This accomplishes:

1. Lower Idle CPU consumption than even Move DotNetty batching scheduling off of DotNetty STEE and onto HashedWheelTimer #4678
2. Significantly lower latencies on systems that don't write heavily
3. No need for users to manually tune their threshold settings in akka.remote.dot-netty.tcp.batching - this is now handled automatically by the FlushConsolidationHandler.

close #4636
close #4563

We still have some more work to do optimizing the DedicatedThreadPool to scale down automatically, but this eliminates most of the CPU noise coming from DotNetty.

[Aaronontheweb]

How It Works

The FlushConsolidationHandler works by batching flushes together, rather than writes, and in this patch we've changed the TcpAssociationHandle to always call IChannel.WriteAndFlushAsync - which creates a 1:1 correlation between writes and flushes.

The algorithm is designed to capture flushes that occur in rapid succession and group them together in order to lower the total number of system calls to the socket, which improves average throughput and decreases CPU utilization.

The flushes are batched together when:

1. The socket is currently performing a read or
2. The total number of flushes is less than DefaultExplicitFlushAfterFlushes or whatever the configured value is - defaults to 30 in Akka.NET.

The flushes are allowed to pass and write out to the socket when:

1. The socket is writing but not reading (immediate write);
2. The total number of flushes is equal to DefaultExplicitFlushAfterFlushes; or
3. A flush has been scheduled onto the EventLoop without being cancelled.

In the third case, we don't use a time-based delay to flush the socket - instead the "flush" call is simply added to the same event queue where all of the read and write events are. It works identically to an actor's mailbox. All of the writes queued up prior to that event get flushed together.

[IgorFedchenko]

LGTM - just left minor note and a question about flushing while reading is in process

[IgorFedchenko]

I guess this should not be changed in this PR, it's just build,cmd updated this file when was running locally?

[Aaronontheweb]

That's correct

[IgorFedchenko]

So when reading is complete we will flush in ChannelReadComplete anyway. But we are trying to flush right here if there are too many flushes are pending? Are we able to flush while reading is in process? Or this is safe?

[Aaronontheweb]

So if a read is sitting in the channel pipeline right now we assume it means that some data is being read by the current application and that might, presumably, be used to produce a response right away - we're trying to batch those writes together into as few flushes as possible and when the read "completes" (we've finished reading all of the data currently inside the buffer) it's safe to flush any writes that are currently pending. This helps reduce latency in lower traffic system AND increases throughput in higher traffic systems.