GC doesn't run when using TransactionScope + async

[karlra]

Description

I have an application that uses the TransactionScope class to control SQL transactions. We also use the same library to do mathematical simulations, but in this case it injects a dummy repository to stop the persistence to SQL. When doing this I noticed that performance became progressively worse, and eventually the app crashed with OOM errors.

I had previously created a bug report here: #1419 but it was closed because it was determined that this is a GC issue and not a TransactionScope issue.

Take this code:

using System;
using System.Diagnostics;
using System.Threading;
using System.Transactions;

namespace TestTrxScope
{
    class Program
    {
        static void Main(string[] args)
        {
            int i = 0;
            double lastElapsed = 0;
            Stopwatch sw = new Stopwatch();
            sw.Start();
            while(true)
            {
                using(var scope = CreateTransactionScope())
                {
                    scope.Complete();
                    i++;
                    if(i % 1_000_000 == 0)
                    {
                        Console.WriteLine($"1m took {sw.Elapsed.TotalSeconds - lastElapsed}");
                        lastElapsed = sw.Elapsed.TotalSeconds;
                        //GC.Collect(2);
                    }
                }
            }
        }

        public static TransactionScope CreateTransactionScope()
        {
            //Memory grows constantly
            return new TransactionScope(TransactionScopeOption.Suppress, new TransactionOptions() { }, TransactionScopeAsyncFlowOption.Enabled);

            //This version doesn't constantly grow in memory usage
            //return new TransactionScope();
        }
    }
}

This program's memory usage will grow forever, and the time it takes to complete 1 million TransactionScopes will also grow forever.

Uncommenting the GC.Collect(2) fixes the ever increasing memory issue, but for some reason the GC does not run on its own in this program. You might think that this is a contrived example but I found this issue in production code, where we have console apps that do extremely heavy calculations.

Note that even if GC.Collect(2) is called manually, the performance per 1 million rounds still gets worse by 75% or so until it evens out. This suggests that our data access code is impacted by this even in situations when the GC might run level 2 collections on its own due to other allocations causing it.

Configuration

.net core 3.1, Windows 10 x64

Regression?

This does not seem to happen on .net framework.

Other information

The situation improves somewhat with .net 5 as I haven't managed to get OOM errors but it still gets massively progressively worse.

I couldn't figure out the best area label to add to this issue. If you have write-permissions please help me learn by adding exactly one area label.

[karlra]

Any chance I could get an update on this? There is a pretty serious leak in TransactionScope with TransactionScopeAsyncFlowOption.Enabled.

[Timovzl]

@karlra, out of curiosity, does omitting TransactionScopeAsyncFlowOption.Enabled affect the problem?

I'm also curious if .NET 7 (current latest) makes any difference.

Finally, what about .NET 7 with <TieredPGO>True</TieredPGO> in the running application's csproj?

[karlra]

Yes, that's the whole point, omitting TransactionScopeAsyncFlowOption.Enabled fixes the issue (but then TransactionScope is unusable with async code).

I just tested with net7 and there is no appreciable difference vs net6. TieredPGO also makes no difference.

This is with TransactionScopeAsyncFlowOption.Enabled. The GC does run, but it doesn't collect whatever is being leaked here:

This is without TransactionScopeAsyncFlowOption.Enabled. GC runs often, but RAM usage is stable.

[roji]

Note similarity with #108762.

Am going to go ahead and close this issue as it has been open for a while, and there's no evidence of a leak: growing memory usage in itself is not a problem, and only means that the GC hasn't yet decided to reclaim that memory. An actual leak would mean that an OutOfMemoryException would be thrown eventually; that has been flagged in #108762 for Android only; if someone can put together a repro that shows an OOM for non-mobile .NET, that would indeed be interesting and we'd want to look into it.

[karlra]

Interesting attitude. And you don't feel like it's a problem that performance gets worse by 800% in this example?

[roji]

@karira first, I put together a quick BenchmarkDotNet test. When filing issues on performance and/or memory issues, please always include such a BenchmarkDotNet benchmark rather than a Stopwatch-based benchmark - that greatly helps ensuring you're seeing correct results and not e.g. getting skewed by warm-up problems (it generally helps getting your issue to be addressed more quickly etc.).

Here are the results (code just below):

BenchmarkDotNet=v0.13.0, OS=macOS 14.6.1 (23G93) [Darwin 23.6.0]
Apple M2 Max, 1 CPU, 12 logical and 12 physical cores
.NET SDK=8.0.203
  [Host]     : .NET 8.0.3 (8.0.324.11423), Arm64 RyuJIT
  DefaultJob : .NET 8.0.3 (8.0.324.11423), Arm64 RyuJIT


|                                  Method |     Mean |   Error |  StdDev |  Gen 0 |  Gen 1 |  Gen 2 | Allocated |
|---------------------------------------- |---------:|--------:|--------:|-------:|-------:|-------:|----------:|
|                                 Default | 164.6 ns | 0.82 ns | 0.77 ns | 0.0746 | 0.0138 |      - |     624 B |
| TransactionScopeAsyncFlowOption_Enabled | 524.1 ns | 6.03 ns | 5.64 ns | 0.0944 | 0.0086 | 0.0038 |     777 B |

Benchmark code

BenchmarkRunner.Run<Benchmark>();

[MemoryDiagnoser]
public class Benchmark
{
    [Benchmark]
    public void Default()
    {
        using (new TransactionScope())
        {
            // Do nothing
        }
    }

    [Benchmark]
    public void TransactionScopeAsyncFlowOption_Enabled()
    {
        using (new TransactionScope(TransactionScopeAsyncFlowOption.Enabled))
        {
            // Do nothing
        }
    }
}

In other words, from TransactionScope is indeed x3.5 slower with TransactionScopeAsyncFlowOption.Enabled, and also allocates more memory, with some of it living long enough to get into gen2. While it may be possible to optimize TransactionScopeAsyncFlowOption.Enabled handling, the benchmark doesn't show a problem per se.

I read through #1419, and if I understand correctly, this issue is basically a duplicate of that, which you opened because #1419 was closed. I essentially agree with what @mconnew wrote there: there's no evidence of a problem with TransactionScope itself - and certainly no memory leak as you've indicated above - although there may be a GC issue lurking here.

I'll reopen and tag this as a GC issue, hopefully someone from the GC team and chime in (I recommend also looking at #1419 as well for previous discussion). Am also happy to try and help if I can.