`Arguments` handling regression in TUnit 0.61.13

[nil4]

A functional regression appears to have been introduced between TUnit 0.58.3 and 0.61.13 where [Arguments] are no longer handled correctly.

Steps to reproduce

Create a repro.cs file with this content:

#:package TUnit@0.61.13

public class Repro
{
    [Test]
    [Arguments(null, 0d, 0d)]
    [Arguments(null, 12d, 12d)]
    [Arguments(false, 42d, 0d)]
    [Arguments(true, 42d, 1d)]
    [Arguments(0, 42d, 0d)]
    [Arguments(1, 42d, 1d)]
    [Arguments(42, 0d, 42d)]
    [Arguments(-1, 0d, -1d)]
    [Arguments(0L, 42d, 0d)]
    [Arguments(1L, 42d, 1d)]
    [Arguments(42L, 0d, 42d)]
    [Arguments(-1L, 0d, -1d)]
    [Arguments(0d, 42d, 0d)]
    [Arguments(1d, 42d, 1d)]
    [Arguments(42d, 0d, 42d)]
    [Arguments(-1d, 0d, -1d)]
    [Arguments(3.5d, 0d, 3.5d)]
    [Arguments(-3.14d, 0d, -3.14d)]
    [Arguments("123", 42d, 123d)]
    [Arguments("-1", 42d, -1d)]
    [Arguments("3.14", 42d, 3.14d)]
    [Arguments("-1.02", 42d, -1.02d)]
    public void DoubleConversion(object? input, double nullValue, double expected)
    { }

    [Test]
    [Arguments(null, Flags.Value1, Flags.Value1)]
    [Arguments(null, Flags.Value2, Flags.Value2)]
    [Arguments(null, Flags.Value1 | Flags.Value2, Flags.Value1 | Flags.Value2)]
    [Arguments(null, Flags.Max, Flags.Max)]
    [Arguments("0", Flags.Max, (Flags)0)]
    [Arguments("1", Flags.Max, Flags.Value1)]
    [Arguments("2", Flags.Max, Flags.Value2)]
    [Arguments("3", Flags.Max, (Flags)3)]
    [Arguments("4", Flags.Max, Flags.Value3)]
    [Arguments("Value1", Flags.Max, Flags.Value1)]
    [Arguments("Value2", Flags.Max, Flags.Value2)]
    [Arguments("value1", Flags.Max, Flags.Value1)]
    [Arguments("value2", Flags.Max, Flags.Value2)]
    [Arguments("Value1,Value2", Flags.Max, Flags.Value1 | Flags.Value2)]
    [Arguments("value1, value2", Flags.Max, Flags.Value1 | Flags.Value2)]
    [Arguments(Flags.Value1, Flags.Max, Flags.Value1)]
    [Arguments(Flags.Value2, Flags.Max, Flags.Value2)]
    [Arguments(Flags.Max, (Flags)0, Flags.Max)]
    public void FlagsEnumConversion(object? input, Flags nullValue, Flags expected)
    { }
}

[Flags]
public enum Flags : byte
{
    Value1 = 1 << 0,
    Value2 = 1 << 1,
    Value3 = 1 << 2,
    Max = 255,
}

Using .NET SDK 10 RC1, run the single-file script: dotnet run repro.cs

Observed behavior

Some (but not all) of the test method invocations are not passed arguments correctly:

> dotnet run repro.cs
failed DoubleConversion(-1.02, 42, -1, 2) (0ms)
  TUnit.Engine.Exceptions.TestFailedException: ArgumentException: Expected exactly 3 arguments, but got 4
failed DoubleConversion(3.14, 42, 3, 14) (0ms)
  TUnit.Engine.Exceptions.TestFailedException: ArgumentException: Expected exactly 3 arguments, but got 4
failed DoubleConversion(3.5, 0, 3, 5) (0ms)
  TUnit.Engine.Exceptions.TestFailedException: ArgumentException: Expected exactly 3 arguments, but got 4
failed DoubleConversion(-3.14, 0, -3, 14) (0ms)
  TUnit.Engine.Exceptions.TestFailedException: ArgumentException: Expected exactly 3 arguments, but got 4

Test run summary: Failed!
  total: 40
  failed: 4
  succeeded: 36
  skipped: 0
  duration: 79ms

Expected behavior

This appears to be a regression, since reverting the TUnit package back to 0.58.3 resolves the issue.

Update repro.cs:

-#:package TUnit@0.61.13
+#:package TUnit@0.58.3

Confirm that all tests compile, run and pass with the previous version:

> dotnet run repro.cs
Test run summary: Passed!
  total: 40
  failed: 0
  succeeded: 40
  skipped: 0
  duration: 71ms

[thomhurst]

I can't seem to replicate this: #3175

[slang25]

passes locally here with dotnet 10 RC1 (macOS)

[nil4]

Perhaps this is related to specific culture settings? Here they are in case it's relevant:

Opening the repro in Rider confirms that the same tests fail there as well:

Reviewing the generated code shows that a few commas made their way in the source code, instead of the . that C# expects for floating-point values, e.g.:

I hope this helps narrow down the issue.

[slang25]

🥇 we have a winner

[nil4]

I also see now that the generated code seems to have lost some (but not all) suffixes on the values in the Arguments list.

Among this particular set of test values, note for instance [Arguments(42L, 0d, 42d)]. The first value is 0L to explicitly denote a long value (this is preserved in the generated code). The next two values, however, have lost their d suffix which was meant to explicitly type them as double (and not some other numeric type, e.g. int)

-[Arguments(42L, 0d, 42d)] // original source
+[Arguments(42L, 0,  42)]  // generated code

The actual test code was elided to keep the repro concise, but depending on what a test method expects, preserving the literal value suffixes in the original code might be important to ensure the correct value is passed-in (e.g. for a boxed object or similar parameter.)

[thomhurst]

Thanks for that insight about culture! I'm just pushing through what should hopefully fix it 😄