Open
Description
Observed a flaky test failure during PR validation.
Microsoft.ML.Tests.OnnxConversionTest.NaiveCalibratorOnnxConversionTest [FAIL]
�[31;1m�[m�[37m Assert.Equal() Failure
�[m�[37m Expected: 0
�[m�[37m Actual: 1
�[m�[30;1m Stack Trace:
�[m�[37m /__w/1/s/test/Microsoft.ML.TestFramework/BaseTestBaseline.cs(201,0): at Microsoft.ML.RunTests.BaseTestBaseline.Done()
�[m�[37m /__w/1/s/test/Microsoft.ML.Tests/OnnxConversionTest.cs(328,0): at Microsoft.ML.Tests.OnnxConversionTest.CommonCalibratorOnnxConversionTest(IEstimator`1 calibrator, IEstimator`1 calibratorNonStandard)
�[m�[37m /__w/1/s/test/Microsoft.ML.Tests/OnnxConversionTest.cs(349,0): at Microsoft.ML.Tests.OnnxConversionTest.NaiveCalibratorOnnxConversionTest()
�[m�[30;1m Output:
�[m�[37m *** Failure #1: Output and baseline mismatch at line .
�[m�[37m Values to compare are 1 and 0.97826087474823
�[m�[37m AllowedVariance: 0.001
�[m�[37m delta: 0.022
�[m�[37m delta2: 0.022
�[m�[37m
�[m�[37m Void Fail(System.String, System.Object[]) /__w/1/s/test/Microsoft.ML.TestFramework/BaseTestBaseline.cs 224
�[m�[37m Boolean CompareNumbersWithTolerance(Double, Double, System.Nullable`1[System.Int32], Int32, Boolean) /__w/1/s/test/Microsoft.ML.TestFramework/BaseTestBaseline.cs 643
�[m�[37m Void CompareScalarValues[T](T, T, Int32) /__w/1/s/test/Microsoft.ML.TestFramework/BaseTestBaseline.cs 787
�[m�[37m Void CompareSelectedColumns[T](System.String, System.String, Microsoft.ML.IDataView, Microsoft.ML.IDataView, Int32, Boolean) /__w/1/s/test/Microsoft.ML.TestFramework/BaseTestBaseline.cs 771
�[m�[37m Void CompareResults(System.String, System.String, Microsoft.ML.IDataView, Microsoft.ML.IDataView, Int32, Boolean) /__w/1/s/test/Microsoft.ML.TestFramework/BaseTestBaseline.cs 698
�[m�[37m Void TestPipeline[TLastTransformer](Microsoft.ML.Data.EstimatorChain`1[TLastTransformer], Microsoft.ML.IDataView, System.String, ColumnComparison[], System.String, System.String) /__w/1/s/test/Microsoft.ML.Tests/OnnxConversionTest.cs 2261
�[m�[37m Void CommonCalibratorOnnxConversionTest(Microsoft.ML.IEstimator`1[Microsoft.ML.ITransformer], Microsoft.ML.IEstimator`1[Microsoft.ML.ITransformer]) /__w/1/s/test/Microsoft.ML.Tests/OnnxConversionTest.cs 315
�[m�[37m Void NaiveCalibratorOnnxConversionTest() /__w/1/s/test/Microsoft.ML.Tests/OnnxConversionTest.cs 349
It passed on rerun. Seems to be either non-deterministic or machine dependent.