list of usage examples that illustrate current problems with ArrayPool and what the desired behavior should be

[Maoni0]

this issue is for folks who want ArrayPool (or other memory pools) to be improved. please explain it with usage examples. a usage example should consist of -

• pseudo code snippet to illustrate your scenario
• description of current problems
• description of your desired behavior, eg, your pool is obviously part of the memory usage in a process, do you want your pool to have an individual threshold that you get notified when it's exceeded and you will do something about it, or do you want to with a default size and leave it up to whenever GC decides to collect.
• any other perf implications you want to mention, eg, does your pool only manage buffers that will need to pinned, what kind of objects do you want to have in this pool, things like byte[] only or any type

then we have a discussion what kinds of behavior our pooling should offer and then we can decide if we need support from the GC and what that would be.

I couldn't figure out the best area label to add to this issue. If you have write-permissions please help me learn by adding exactly one area label.

[jkotas]

Problem 1: ArrayPool does not ever release pooled memory, unless there is a memory pressure.

using System;
using System.Buffers;

DoStuff<int>();
DoStuff<uint>();
DoStuff<long>();
DoStuff<ulong>();
DoStuff<DateTime>();
DoStuff<DateTimeOffset>();

// Let's say the above was done during startup and the rest of the application won't use those ArrayPools again

Console.WriteLine(GC.GetTotalMemory(true));

void DoStuff<T>()
{
   ArrayPool<T>.Shared.Return(ArrayPool<T>.Shared.Rent(1000000));
}

Actual behavior: Array pools are keeping 50MB forever
Expected behavior: The memory is released after it is not used for a while

dotnet/runtime libraries are reluctant to use ArrayPools for anything but byte and char to avoid hitting this issue.

[jkotas]

Problem 2: Array pool does not pool enough

using System;
using System.Buffers;

for (int i = 0; i < 1000000; i++)
    Nested(100);

Console.WriteLine(GC.CollectionCount(0));

void Nested(int depth)
{
   if (depth == 0) return;

   var a = ArrayPool<byte>.Shared.Rent(10);
   Nested(depth-1);
   ArrayPool<byte>.Shared.Return(a);
}

Expected: 0 or very few GCs
Actual: Hundreds of GCs

We have to watch for using the array pool too much that would lead to hitting the threasholds. The Array pool should dynamically resize based on the application behavior.

[jkotas]

Problem 3: ArrayPool does not work for very large arrays

using System;
using System.Buffers;

for (int i = 0; i < 1000; i++)
   ArrayPool<byte>.Shared.Return(ArrayPool<byte>.Shared.Rent(10_000_000));

Console.WriteLine(GC.CollectionCount(2));

Expected: 0 or very few GCs
Actual: Hundred of Gen2 GCs

This implementation limitation largely exists to reduce severity of Problem 1.

A side-effect is that code tends to use array pool limits for buffer sizes instead of figuring about what the rigth buffer size should actually be. Example from this week: https://github.com/dotnet/runtime/pull/49304/files#diff-7721d7d6b3bf50b13337accfc3a3857f235a3901e99c6f0a9ff4c2c5f84c2784R49

[jkotas]

cc @davidfowl

Tagging subscribers to this area: @tannergooding
See info in area-owners.md if you want to be subscribed.

Issue Details

---

this issue is for folks who want ArrayPool (or other memory pools) to be improved. please explain it with usage examples. a usage example should consist of -

• pseudo code snippet to illustrate your scenario
• description of current problems
• description of your desired behavior, eg, your pool is obviously part of the memory usage in a process, do you want your pool to have an individual threshold that you get notified when it's exceeded and you will do something about it, or do you want to with a default size and leave it up to whenever GC decides to collect.
• any other perf implications you want to mention, eg, does your pool only manage buffers that will need to pinned, what kind of objects do you want to have in this pool, things like byte[] only or any type

then we have a discussion what kinds of behavior our pooling should offer and then we can decide if we need support from the GC and what that would be.

Author:	Maoni0
Assignees:	-
Labels:	area-System.Runtime, tenet-performance
Milestone:	Future

[Maoni0]

thanks @jkotas! I will take a look at these hopefully in the next couple of weeks.

[antonfirsov]

Important aspect of Problem 1 (#52098 (comment)) is that ArrayPools are per-type. A 3rd library party may unintentionally retain a bunch of memory by using ArrayPool<SomeLargeUniqueStruct>.Shared (this actually happened in an early alpha of ImageSharp).