Open
Description
Run Information
| Name | Value |
|---|---|
| Architecture | x64 |
| OS | ubuntu 18.04 |
| Queue | TigerUbuntu |
| Baseline | 84fd85932fd3b371cad69cc2412404ea50158c63 |
| Compare | 18321c608e32bc5d13ad0299822cf02ab7ebfd6c |
| Diff | Diff |
| Configs | CompilationMode:wasm, RunKind:micro |
Improvements in System.Numerics.Tests.Perf_Matrix4x4
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Numerics.Tests.Perf_Matrix4x4*'Payloads
System.Numerics.Tests.Perf_Matrix4x4.NegateBenchmark
ETL Files
Histogram
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 43.431854634049536 < 75.38529831169768.
IsChangePoint: Marked as a change because one of 5/19/2023 3:32:16 PM, 7/5/2023 10:47:48 AM, 7/10/2023 9:23:13 PM falls between 7/1/2023 10:14:52 PM and 7/10/2023 9:23:13 PM.
IsImprovementStdDev: Marked as improvement because 72.81406111356806 (T) = (0 -41.17699581925175) / Math.Sqrt((44.68482732817432 / (299)) + (1.6239963781739284 / (8))) is greater than 1.967772354556772 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (8) - 2, .975) and 0.5121481114688994 = (84.40470722217306 - 41.17699581925175) / 84.40470722217306 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
System.Numerics.Tests.Perf_Matrix4x4.EqualityOperatorBenchmark
ETL Files
Histogram
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 117.77940690243939 < 146.24206507746774.
IsChangePoint: Marked as a change because one of 7/5/2023 10:47:48 AM, 7/10/2023 9:23:13 PM falls between 7/1/2023 10:14:52 PM and 7/10/2023 9:23:13 PM.
IsImprovementStdDev: Marked as improvement because 53.385068596375405 (T) = (0 -119.15339966185641) / Math.Sqrt((37.910732015952846 / (299)) + (2.9138978675352787 / (8))) is greater than 1.967772354556772 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (8) - 2, .975) and 0.23893872976682967 = (156.56216433842542 - 119.15339966185641) / 156.56216433842542 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
System.Numerics.Tests.Perf_Matrix4x4.MultiplyByScalarOperatorBenchmark
ETL Files
Histogram
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 62.91435086425536 < 83.42833483927521.
IsChangePoint: Marked as a change because one of 3/25/2023 7:34:08 PM, 3/26/2023 7:47:31 AM, 5/19/2023 3:32:16 PM, 7/5/2023 10:47:48 AM, 7/10/2023 9:23:13 PM falls between 7/1/2023 10:14:52 PM and 7/10/2023 9:23:13 PM.
IsImprovementStdDev: Marked as improvement because 39.54789097145629 (T) = (0 -65.319163066207) / Math.Sqrt((27.31009890861715 / (299)) + (2.8868977496118666 / (8))) is greater than 1.967772354556772 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (8) - 2, .975) and 0.28934073596792176 = (91.91347579936505 - 65.319163066207) / 91.91347579936505 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
System.Numerics.Tests.Perf_Matrix4x4.MultiplyByScalarBenchmark
ETL Files
Histogram
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 63.63384314343538 < 83.70605013120051.
IsChangePoint: Marked as a change because one of 3/21/2023 12:39:01 PM, 5/20/2023 7:49:01 PM, 7/5/2023 10:47:48 AM, 7/10/2023 9:23:13 PM falls between 7/1/2023 10:14:52 PM and 7/10/2023 9:23:13 PM.
IsImprovementStdDev: Marked as improvement because 14.215643210672953 (T) = (0 -67.3993637378321) / Math.Sqrt((26.433336103953877 / (299)) + (23.705898695311706 / (8))) is greater than 1.967772354556772 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (8) - 2, .975) and 0.26924578819852624 = (92.23260386235405 - 67.3993637378321) / 92.23260386235405 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
System.Numerics.Tests.Perf_Matrix4x4.SubtractBenchmark
ETL Files
Histogram
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 55.32811161988938 < 106.93671230814905.
IsChangePoint: Marked as a change because one of 5/19/2023 8:24:48 PM, 7/5/2023 10:47:48 AM, 7/10/2023 9:23:13 PM falls between 7/1/2023 10:14:52 PM and 7/10/2023 9:23:13 PM.
IsImprovementStdDev: Marked as improvement because 107.25268977877302 (T) = (0 -54.70392643792112) / Math.Sqrt((37.50712157337245 / (299)) + (1.628289900381673 / (8))) is greater than 1.967772354556772 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (8) - 2, .975) and 0.529308892945681 = (116.22043760348363 - 54.70392643792112) / 116.22043760348363 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
System.Numerics.Tests.Perf_Matrix4x4.CreateTranslationFromScalarXYZ
ETL Files
Histogram
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 23.752296983493633 < 36.95115303388243.
IsChangePoint: Marked as a change because one of 5/13/2023 2:42:33 PM, 5/14/2023 7:30:23 PM, 7/5/2023 10:47:48 AM, 7/10/2023 9:23:13 PM falls between 7/1/2023 10:14:52 PM and 7/10/2023 9:23:13 PM.
IsImprovementStdDev: Marked as improvement because 18.300784731422727 (T) = (0 -23.64994760878155) / Math.Sqrt((55.511444665753686 / (299)) + (5.760510948949717 / (8))) is greater than 1.967772354556772 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (8) - 2, .975) and 0.4241087305126068 = (41.06668890784299 - 23.64994760878155) / 41.06668890784299 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
System.Numerics.Tests.Perf_Matrix4x4.GetDeterminantBenchmark
ETL Files
Histogram
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 52.479624403106186 < 62.9980474587766.
IsChangePoint: Marked as a change because one of 5/16/2023 7:34:13 AM, 7/5/2023 10:47:48 AM, 7/10/2023 9:23:13 PM falls between 7/1/2023 10:14:52 PM and 7/10/2023 9:23:13 PM.
IsImprovementStdDev: Marked as improvement because 27.29220548659031 (T) = (0 -53.84381561064058) / Math.Sqrt((29.404914537781327 / (299)) + (1.3816883092992194 / (8))) is greater than 1.967772354556772 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (8) - 2, .975) and 0.20879524043437328 = (68.05294705279701 - 53.84381561064058) / 68.05294705279701 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
System.Numerics.Tests.Perf_Matrix4x4.CreateScaleFromScalarBenchmark
ETL Files
Histogram
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 23.40949093405259 < 36.44548843004495.
IsChangePoint: Marked as a change because one of 7/5/2023 10:47:48 AM, 7/10/2023 9:23:13 PM falls between 7/1/2023 10:14:52 PM and 7/10/2023 9:23:13 PM.
IsImprovementStdDev: Marked as improvement because 29.59227058874679 (T) = (0 -22.822944390180993) / Math.Sqrt((60.06634095152997 / (299)) + (1.5557174458442151 / (8))) is greater than 1.967772354556772 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (8) - 2, .975) and 0.44911720522483567 = (41.429764382995266 - 22.822944390180993) / 41.429764382995266 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
System.Numerics.Tests.Perf_Matrix4x4.SubtractOperatorBenchmark
ETL Files
Histogram
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 57.49394723226845 < 106.34782815396066.
IsChangePoint: Marked as a change because one of 5/19/2023 3:32:16 PM, 7/5/2023 10:47:48 AM, 7/10/2023 9:23:13 PM falls between 7/1/2023 10:14:52 PM and 7/10/2023 9:23:13 PM.
IsImprovementStdDev: Marked as improvement because 62.84091780503685 (T) = (0 -55.95924280042281) / Math.Sqrt((52.25116839604226 / (299)) + (6.04467384696511 / (8))) is greater than 1.967772354556772 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (8) - 2, .975) and 0.5199590800702983 = (116.57181810379333 - 55.95924280042281) / 116.57181810379333 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
System.Numerics.Tests.Perf_Matrix4x4.AddOperatorBenchmark
ETL Files
Histogram
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 57.01239442411866 < 106.25541829560296.
IsChangePoint: Marked as a change because one of 5/19/2023 3:32:16 PM, 7/5/2023 10:47:48 AM, 7/10/2023 9:23:13 PM falls between 7/1/2023 10:14:52 PM and 7/10/2023 9:23:13 PM.
IsImprovementStdDev: Marked as improvement because 82.11402514739119 (T) = (0 -54.86834423002084) / Math.Sqrt((39.912047526632115 / (299)) + (3.4231168664774034 / (8))) is greater than 1.967772354556772 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (8) - 2, .975) and 0.5285910095301118 = (116.39223124558892 - 54.86834423002084) / 116.39223124558892 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
System.Numerics.Tests.Perf_Matrix4x4.AddBenchmark
ETL Files
Histogram
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 57.32674398713448 < 106.97071638348322.
IsChangePoint: Marked as a change because one of 5/19/2023 10:55:27 PM, 7/5/2023 10:47:48 AM, 7/10/2023 9:23:13 PM falls between 7/1/2023 10:14:52 PM and 7/10/2023 9:23:13 PM.
IsImprovementStdDev: Marked as improvement because 91.25171092547622 (T) = (0 -55.15027593764676) / Math.Sqrt((50.133412119392915 / (299)) + (2.273983215020368 / (8))) is greater than 1.967772354556772 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (8) - 2, .975) and 0.5265831409752223 = (116.49411060530124 - 55.15027593764676) / 116.49411060530124 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
System.Numerics.Tests.Perf_Matrix4x4.InequalityOperatorBenchmark
ETL Files
Histogram
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 126.60689562664507 < 154.94628762505238.
IsChangePoint: Marked as a change because one of 5/25/2023 10:38:38 PM, 7/5/2023 10:47:48 AM, 7/10/2023 9:23:13 PM falls between 7/1/2023 10:14:52 PM and 7/10/2023 9:23:13 PM.
IsImprovementStdDev: Marked as improvement because 17.269437406824483 (T) = (0 -127.70532880546261) / Math.Sqrt((60.89353497025987 / (299)) + (34.379583610833194 / (8))) is greater than 1.967772354556772 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (8) - 2, .975) and 0.22293812979977368 = (164.3438363184088 - 127.70532880546261) / 164.3438363184088 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
System.Numerics.Tests.Perf_Matrix4x4.IsIdentityBenchmark
ETL Files
Histogram
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 106.87407840062777 < 124.77804106663325.
IsChangePoint: Marked as a change because one of 5/19/2023 3:32:16 PM, 7/5/2023 10:47:48 AM, 7/10/2023 9:23:13 PM falls between 7/1/2023 10:14:52 PM and 7/10/2023 9:23:13 PM.
IsImprovementStdDev: Marked as improvement because 51.03124078746893 (T) = (0 -103.63188172384194) / Math.Sqrt((81.10456959409406 / (299)) + (2.240416824397212 / (8))) is greater than 1.967772354556772 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (8) - 2, .975) and 0.2677358018886962 = (141.52252969779897 - 103.63188172384194) / 141.52252969779897 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
System.Numerics.Tests.Perf_Matrix4x4.IdentityBenchmark
ETL Files
Histogram
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 22.69764140416124 < 36.22840769639247.
IsChangePoint: Marked as a change because one of 3/23/2023 3:52:33 AM, 3/24/2023 2:55:56 PM, 7/5/2023 10:47:48 AM, 7/10/2023 9:23:13 PM falls between 7/1/2023 10:14:52 PM and 7/10/2023 9:23:13 PM.
IsImprovementStdDev: Marked as improvement because 30.144495289654014 (T) = (0 -23.173662148645) / Math.Sqrt((37.49062835398752 / (299)) + (1.7126562926274485 / (8))) is greater than 1.967772354556772 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (8) - 2, .975) and 0.431140185351725 = (40.73703494590004 - 23.173662148645) / 40.73703494590004 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
System.Numerics.Tests.Perf_Matrix4x4.TranslationBenchmark
ETL Files
Histogram
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 33.25276453476744 < 43.44076798936607.
IsChangePoint: Marked as a change because one of 5/15/2023 9:18:21 AM, 5/16/2023 7:34:13 AM, 7/5/2023 10:47:48 AM, 7/10/2023 9:23:13 PM falls between 7/1/2023 10:14:52 PM and 7/10/2023 9:23:13 PM.
IsImprovementStdDev: Marked as improvement because 17.42180202521696 (T) = (0 -34.255291711090095) / Math.Sqrt((41.27510718286882 / (299)) + (4.636117052286339 / (8))) is greater than 1.967772354556772 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (8) - 2, .975) and 0.3010991937359207 = (49.01309514035208 - 34.255291711090095) / 49.01309514035208 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
System.Numerics.Tests.Perf_Matrix4x4.Transpose
ETL Files
Histogram
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 56.519363954381845 < 70.78913546395304.
IsChangePoint: Marked as a change because one of 5/19/2023 3:32:16 PM, 7/5/2023 10:47:48 AM, 7/10/2023 9:23:13 PM falls between 7/1/2023 10:14:52 PM and 7/10/2023 9:23:13 PM.
IsImprovementStdDev: Marked as improvement because 44.4960809348505 (T) = (0 -56.214998896906444) / Math.Sqrt((40.78194311641916 / (299)) + (0.8858078826558244 / (8))) is greater than 1.967772354556772 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (8) - 2, .975) and 0.28237276439185227 = (78.33453930893171 - 56.214998896906444) / 78.33453930893171 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
System.Numerics.Tests.Perf_Matrix4x4.NegationOperatorBenchmark
ETL Files
Histogram
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 40.527551550193095 < 77.52892865991303.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 7/5/2023 10:47:48 AM, 7/10/2023 9:23:13 PM falls between 7/1/2023 10:14:52 PM and 7/10/2023 9:23:13 PM.
IsImprovementStdDev: Marked as improvement because 42.66240209983075 (T) = (0 -41.32385206760178) / Math.Sqrt((38.15913445884302 / (299)) + (7.052285659134663 / (8))) is greater than 1.967772354556772 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (8) - 2, .975) and 0.5091080994758639 = (84.18116498455034 - 41.32385206760178) / 84.18116498455034 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
| Name | Value |
|---|---|
| Architecture | x64 |
| OS | ubuntu 18.04 |
| Queue | TigerUbuntu |
| Baseline | 84fd85932fd3b371cad69cc2412404ea50158c63 |
| Compare | 18321c608e32bc5d13ad0299822cf02ab7ebfd6c |
| Diff | Diff |
| Configs | CompilationMode:wasm, RunKind:micro |
Improvements in System.Numerics.Tests.Perf_Vector2
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio |
|---|---|---|---|---|---|---|---|---|
| TransformByMatrix4x4Benchmark - Duration of single invocation | 73.83 ns | 55.33 ns | 0.75 | 0.28 | False | |||
| TransformNormalByMatrix4x4Benchmark - Duration of single invocation | 63.41 ns | 53.71 ns | 0.85 | 0.35 | False |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Numerics.Tests.Perf_Vector2*'Payloads
System.Numerics.Tests.Perf_Vector2.TransformByMatrix4x4Benchmark
ETL Files
Histogram
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 55.32518353476052 < 70.80272223121075.
IsChangePoint: Marked as a change because one of 3/21/2023 4:52:34 PM, 7/5/2023 10:47:48 AM, 7/10/2023 9:23:13 PM falls between 7/1/2023 10:14:52 PM and 7/10/2023 9:23:13 PM.
IsImprovementStdDev: Marked as improvement because 27.654154237666525 (T) = (0 -56.73384209213503) / Math.Sqrt((54.4563692332083 / (299)) + (1.951672168630068 / (8))) is greater than 1.967772354556772 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (8) - 2, .975) and 0.24137599815552657 = (74.78519260423573 - 56.73384209213503) / 74.78519260423573 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
System.Numerics.Tests.Perf_Vector2.TransformNormalByMatrix4x4Benchmark
ETL Files
Histogram
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 53.70833419798295 < 62.58863367625987.
IsChangePoint: Marked as a change because one of 3/23/2023 12:23:24 PM, 3/24/2023 3:46:52 AM, 7/5/2023 10:47:48 AM, 7/10/2023 9:23:13 PM falls between 7/1/2023 10:14:52 PM and 7/10/2023 9:23:13 PM.
IsImprovementStdDev: Marked as improvement because 16.367895728391673 (T) = (0 -55.01807959856332) / Math.Sqrt((47.02017843986054 / (299)) + (3.8540101427260356 / (8))) is greater than 1.967772354556772 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (8) - 2, .975) and 0.19212547558942206 = (68.10225837818601 - 55.01807959856332) / 68.10225837818601 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
| Name | Value |
|---|---|
| Architecture | x64 |
| OS | ubuntu 18.04 |
| Queue | TigerUbuntu |
| Baseline | 84fd85932fd3b371cad69cc2412404ea50158c63 |
| Compare | 18321c608e32bc5d13ad0299822cf02ab7ebfd6c |
| Diff | Diff |
| Configs | CompilationMode:wasm, RunKind:micro |
Improvements in System.Numerics.Tests.Perf_Vector4
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio |
|---|---|---|---|---|---|---|---|---|
| DistanceSquaredJitOptimizeCanaryBenchmark - Duration of single invocation | 19.02 ns | 15.34 ns | 0.81 | 0.51 | False | |||
| TransformVector2ByMatrix4x4Benchmark - Duration of single invocation | 72.99 ns | 60.06 ns | 0.82 | 0.33 | False |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Numerics.Tests.Perf_Vector4*'Payloads
System.Numerics.Tests.Perf_Vector4.DistanceSquaredJitOptimizeCanaryBenchmark
ETL Files
Histogram
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 15.339535433831243 < 17.94887644588655.
IsChangePoint: Marked as a change because one of 7/5/2023 10:47:48 AM, 7/10/2023 9:23:13 PM falls between 7/1/2023 10:14:52 PM and 7/10/2023 9:23:13 PM.
IsImprovementStdDev: Marked as improvement because 14.911854657430458 (T) = (0 -15.174032674905462) / Math.Sqrt((4.015210968851586 / (299)) + (0.8082272294158553 / (8))) is greater than 1.967772354556772 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (8) - 2, .975) and 0.24951395168105242 = (20.218940390556973 - 15.174032674905462) / 20.218940390556973 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
System.Numerics.Tests.Perf_Vector4.TransformVector2ByMatrix4x4Benchmark
ETL Files
Histogram
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 60.063510069769684 < 69.76520771613414.
IsChangePoint: Marked as a change because one of 3/23/2023 3:52:33 AM, 7/5/2023 10:47:48 AM, 7/10/2023 9:23:13 PM falls between 7/1/2023 10:14:52 PM and 7/10/2023 9:23:13 PM.
IsImprovementStdDev: Marked as improvement because 18.384537666917645 (T) = (0 -57.19700215845097) / Math.Sqrt((42.51747007687485 / (299)) + (6.376618003395749 / (8))) is greater than 1.967772354556772 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (8) - 2, .975) and 0.23752182005640612 = (75.01460850024883 - 57.19700215845097) / 75.01460850024883 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
| Name | Value |
|---|---|
| Architecture | x64 |
| OS | ubuntu 18.04 |
| Queue | TigerUbuntu |
| Baseline | 84fd85932fd3b371cad69cc2412404ea50158c63 |
| Compare | 18321c608e32bc5d13ad0299822cf02ab7ebfd6c |
| Diff | Diff |
| Configs | CompilationMode:wasm, RunKind:micro |
Improvements in System.Numerics.Tests.Perf_VectorOf<Single>
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio |
|---|---|---|---|---|---|---|---|---|
| OneBenchmark - Duration of single invocation | 22.23 ns | 12.24 ns | 0.55 | 0.40 | False |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Numerics.Tests.Perf_VectorOf<Single>*'Payloads
System.Numerics.Tests.Perf_VectorOf<Single>.OneBenchmark
ETL Files
Histogram
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 12.239813438035426 < 22.17392703819603.
IsChangePoint: Marked as a change because one of 7/5/2023 10:47:48 AM, 7/10/2023 9:23:13 PM falls between 7/1/2023 10:14:52 PM and 7/10/2023 9:23:13 PM.
IsImprovementStdDev: Marked as improvement because 15.842133640911072 (T) = (0 -12.820211461796726) / Math.Sqrt((7.948649913939041 / (299)) + (4.228782777877843 / (8))) is greater than 1.967772354556772 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (8) - 2, .975) and 0.4793668365867695 = (24.624269759821708 - 12.820211461796726) / 24.624269759821708 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository