Array might not be zero-initialized

[threbejk]

Description

We have found, that elements of an array might not be always initialized to the default value. For example, after calling new double[size], some elements might be nonzero. We have started noticing this problem after upgrading to .net7. It seems, that this happens especially when the GC heap size limit is set. (Initially, we observed this with 8 GB limit.)

Reproduction Steps

Run this C# program:

using System;

var rand = new Random();
try
{
    for (int k = 0; k < 1_000_000; k++)
    {
        Test(k);
    }
    Console.Write("Succeeded");
}
catch (Exception ex)
{
    Console.WriteLine("Failed");
    Console.WriteLine(ex.Message);    
}

void Test(int iteration)
{
    int size = (int)Math.Pow(10, 5 + rand.NextDouble() * (1 + rand.NextDouble()));

    var m = new double[size];
    for (int i = 0; i < size; i++)
        if (m[i] != 0)
            throw new Exception(m[i].ToString() + "; " + i + "/" + size + "; iteration " + iteration.ToString());

    for (int i = 0; i < size; i++)
        m[i] = 21;
}

With GC HeapHardLimit set to 1 GB

{
  "configProperties": {
    "System.GC.HeapHardLimit": 1000000000
  }
}

Expected behavior

We expect that Exception is not thrown, because elements of the array should be initialized to zero.

Actual behavior

With high probability, Exception is thrown, because an element is not zero.

Regression?

We haven't noticed this issue in .net6.

Known Workarounds

No response

Configuration

.net version 7.0.100
MS Windows 10 Pro, x64

Other information

No response

Tagging subscribers to this area: @dotnet/gc
See info in area-owners.md if you want to be subscribed.

Issue Details

---

Description

We have found, that elements of an array might not be always initialized to the default value. For example, after calling new double[size], some elements might be nonzero. We have started noticing this problem after upgrading to .net7. It seems, that this happens especially when the GC heap size limit is set. (Initially, we observed this with 8 GB limit.)

Reproduction Steps

Run this C# program:

using System;

var rand = new Random();
try
{
    for (int k = 0; k < 1_000_000; k++)
    {
        Test(k);
    }
    Console.Write("Succeeded");
}
catch (Exception ex)
{
    Console.WriteLine("Failed");
    Console.WriteLine(ex.Message);    
}

void Test(int iteration)
{
    int size = (int)Math.Pow(10, 5 + rand.NextDouble() * (1 + rand.NextDouble()));

    var m = new double[size];
    for (int i = 0; i < size; i++)
        if (m[i] != 0)
            throw new Exception(m[i].ToString() + "; " + i + "/" + size + "; iteration " + iteration.ToString());

    for (int i = 0; i < size; i++)
        m[i] = 21;
}

With GC HeapHardLimit set to 1 GB

{
  "configProperties": {
    "System.GC.HeapHardLimit": 1000000000
  }
}

Expected behavior

We expect that Exception is not thrown, because elements of the array should be initialized to zero.

Actual behavior

With high probability, Exception is thrown, because an element is not zero.

Regression?

We haven't noticed this issue in .net6.

Known Workarounds

No response

Configuration

.net version 7.0.100
MS Windows 10 Pro, x64

Other information

No response

Author:	threbejk
Assignees:	-
Labels:	area-GC-coreclr
Milestone:	-

[vcsjones]

Anecdotally, I can reproduce this on Windows 11 ARM. I cannot reproduce this on macOS with Apple Silicon.

[mangod9]

Thanks for reporting this issue. Unfortunately, its not reproing on my machine. Are you running with WKS or SVR GC? Would it be possible to share a dump when its non-zero (calling Debugger.Break instead of throwing an exception) and using .dump -ma to generate a full memory dump). Thanks!

@cshung @PeterSolMS fyi.

[vcsjones]

@mangod9 Dump is too big for GitHub even zipped, so attached here. https://1drv.ms/u/s!AqFBIHiaPFEtk0Q6Wf5oOTFCj1KW?e=kN6uHm

I used a slightly modified version of the code, which is below.

using System;
using System.Diagnostics;

Console.WriteLine(Environment.ProcessId);
var rand = new Random();
for (int k = 0; k < 1_000_000; k++)
{
    Test(k);
}
Console.Write("Succeeded");


void Test(int iteration)
{
    int size = (int)Math.Pow(10, 5 + rand.NextDouble() * (1 + rand.NextDouble()));

    var m = new double[size];
    for (int i = 0; i < size; i++)
    {
        if (m[i] != 0)
        {
            throw new Exception(m[i].ToString() + "; " + i + "/" + size + "; iteration " + iteration.ToString());
        }
    }

    for (int i = 0; i < size; i++)
    {
        m[i] = 21;
    }
}

With a runtimeconfig.template.json:

{
  "configProperties": {
    "System.GC.HeapHardLimit": 1000000000
  }
}

Are you running with WKS or SVR GC?

This does not reproduce when I use the Server GC. When I run with "System.GC.Server": true in the runtime config, it does not fail. When I use workstation GC, it fails for me in a couple of seconds.

[HonzaMD]

Hi, its WorkstationGC, Console App, all default settings, except the System.GC.HeapHardLimit.
I tried to create a dump but its compressed size is bigger than this form allows.

Than you, @vcsjones

[mangod9]

thanks @vcsjones / @HonzaMD, does it repro without the array item assignment in the second for loop? From the exception in the dump I notice that the message is 21; 0/8457670; iteration 572 meaning its possibly getting assigned out of order? Or do you notice it failing with other values as well?

[vcsjones]

@mangod9

does it repro without the array item assignment in the second for loop?

Yes.

Result:

Unhandled exception. System.Exception: 1.1401181653923E-311; 1962135/4542241; iteration 722
   at Program.<<Main>$>g__Test|0_0(Int32 iteration, <>c__DisplayClass0_0&) in C:\code\scratch\Program.cs:line 22
   at Program.<Main>$(String[] args) in C:\code\scratch\Program.cs:line 8

[vcsjones]

It just seems to be some uninitialized value.

[vcsjones]

Also of note: I can still reproduce this with set COMPlus_JITMinOpts=1 and with a debug build.

[mangod9]

ok, could you please share the dump for that case then? The one you shared has the array elements all set to 21, which is weird too. I want to check if only a few elements are uninitialized within it (most likely some heap corruption under low memory conditions)

[vcsjones]

@mangod9 https://1drv.ms/u/s!AqFBIHiaPFEtk0Uw3R1wx0oNckIS?e=TNpBHb (I zipped it this time).