ResponseCachingMiddleware: exception "Cannot allocate more than 4096 bytes in a single buffer" when a client disconnects

[xhafan]

When we upgraded from .NET Core 3.1 to .NET 5.0 (5.0.3), we started getting rare random exception from ResponseCachingMiddleware:

System.ArgumentOutOfRangeException: Cannot allocate more than 4096 bytes in a single buffer (Parameter 'size')
at System.Buffers.MemoryPoolThrowHelper.ThrowArgumentOutOfRangeException_BufferRequestTooLarge(Int32 maxSize)
at System.Buffers.SlabMemoryPool.Rent(Int32 size)
at Microsoft.AspNetCore.Server.Kestrel.Core.Internal.Http.Http1OutputProducer.GetSpan(Int32 sizeHint)
at Microsoft.AspNetCore.Server.Kestrel.Core.Internal.Http.HttpProtocol.GetSpan(Int32 sizeHint)
at Microsoft.AspNetCore.Server.Kestrel.Core.Internal.Http.HttpResponsePipeWriter.GetSpan(Int32 sizeHint)
at Microsoft.AspNetCore.ResponseCaching.CachedResponseBody.CopyToAsync(PipeWriter destination, CancellationToken cancellationToken)
at Microsoft.AspNetCore.ResponseCaching.ResponseCachingMiddleware.TryServeCachedResponseAsync(ResponseCachingContext context, IResponseCacheEntry cache
Entry)
at Microsoft.AspNetCore.ResponseCaching.ResponseCachingMiddleware.TryServeFromCacheAsync(ResponseCachingContext context)
at Microsoft.AspNetCore.ResponseCaching.ResponseCachingMiddleware.Invoke(HttpContext httpContext)
at ResponseCachingMiddleware.DotNet5.Bug.RequestLoggingMiddleware.Invoke(HttpContext httpContext) in C:\Projects\ResponseCachingMiddleware.DotNet5.Bug\
ResponseCachingMiddleware.DotNet5.Bug\RequestLoggingMiddleware.cs:line 22
at Microsoft.AspNetCore.Server.Kestrel.Core.Internal.Http.HttpProtocol.ProcessRequests[TContext] (IHttpApplication`1 application)

We get this exception once in 3 days, very random and very rare. I discovered it is likely happening when a client disconnects in the middle of downloading a large cached response. Here is a demo project with a clean Web API project to replicate it: https://github.com/xhafan/ResponseCachingMiddleware.DotNet5.Bug

I can replicate it on Amazon Linux (amzn2-ami-ecs-hvm-2.0.20190614-x86_64-ebs) and on Windows 7 SP1 with latest .NET 5.0.3 . This issues was not happening on .NET Core 3.1

Thanks for contacting us.
We're moving this issue to the Next sprint planning milestone for future evaluation / consideration. We will evaluate the request when we are planning the work for the next milestone. To learn more about what to expect next and how this issue will be handled you can read more about our triage process here.

[BrennanConroy]

Triage: We should check Http2 as well

[xhafan]

Since I reported this issue, we got around 1400 exceptions like this in production, so it's not a rare issue really.

[toiyabe]

Anyone have a work around for this? Our application gets this regularly... by that I mean thousands of times a day. Is there a way to set the limit higher?

[halter73]

It looks like Http1/2/3OutputProducer all have a bug in GetFakeMemory where it fails with "sizeHints" (should be named "minSize" but isn't for historical reasons) greater than 4KB. We should do the same thing as StreamPipeWriter and fall back to the ArrayPool for large "sizeHints" or just new up a byte array like Http1OutputProducer.AddSegment does.