Description
Run Information
Architecture | arm64 |
---|---|
OS | ubuntu 20.04 |
Baseline | 3c47b2c75dfca5a82a59db238f103cdbda902826 |
Compare | 873775b42431bf27a9ae69633864e5290443c29b |
Diff | Diff |
Regressions in System.Numerics.Tests.Perf_Quaternion
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
---|---|---|---|---|---|---|---|---|---|---|
MultiplyByQuaternionOperatorBenchmark - Duration of single invocation | 2.22 ns | 7.94 ns | 3.57 | 0.36 | False | |||||
LerpBenchmark - Duration of single invocation | 14.14 ns | 36.98 ns | 2.62 | 0.01 | False | |||||
SlerpBenchmark - Duration of single invocation | 14.12 ns | 37.19 ns | 2.63 | 0.03 | False | |||||
CreateFromRotationMatrixBenchmark - Duration of single invocation | 15.51 ns | 17.74 ns | 1.14 | 0.05 | False | |||||
DivisionOperatorBenchmark - Duration of single invocation | 8.90 ns | 15.65 ns | 1.76 | 0.01 | False | |||||
ConcatenateBenchmark - Duration of single invocation | 2.20 ns | 8.08 ns | 3.68 | 0.39 | False |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
Payloads
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Numerics.Tests.Perf_Quaternion*'
Payloads
Histogram
System.Numerics.Tests.Perf_Quaternion.MultiplyByQuaternionOperatorBenchmark
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 7.941581479533306 > 2.3212239364452296.
IsChangePoint: Marked as a change because one of 2/2/2023 2:17:49 AM, 2/7/2023 2:48:42 AM falls between 1/29/2023 12:56:15 AM and 2/7/2023 2:48:42 AM.
IsRegressionStdDev: Marked as regression because -167.45954392890064 (T) = (0 -8.105748291406591) / Math.Sqrt((0.009554647809051419 / (24)) + (0.013660284774689476 / (16))) is less than -2.0243941639098457 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (24) + (16) - 2, .025) and -2.7170171732207136 = (2.180713166945941 - 8.105748291406591) / 2.180713166945941 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Numerics.Tests.Perf_Quaternion.LerpBenchmark
```log
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 36.983729453513575 > 14.878670792138404.
IsChangePoint: Marked as a change because one of 2/2/2023 2:17:49 AM, 2/7/2023 2:48:42 AM falls between 1/29/2023 12:56:15 AM and 2/7/2023 2:48:42 AM.
IsRegressionStdDev: Marked as regression because -655.3871681404464 (T) = (0 -37.14380633354826) / Math.Sqrt((0.014716382798237213 / (24)) + (0.009806419615714195 / (16))) is less than -2.0243941639098457 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (24) + (16) - 2, .025) and -1.6166620942734653 = (14.195110027709367 - 37.14380633354826) / 14.195110027709367 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Numerics.Tests.Perf_Quaternion.SlerpBenchmark
```log
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 37.194904381975675 > 14.831858502405032.
IsChangePoint: Marked as a change because one of 2/2/2023 2:17:49 AM, 2/7/2023 2:48:42 AM falls between 1/29/2023 12:56:15 AM and 2/7/2023 2:48:42 AM.
IsRegressionStdDev: Marked as regression because -681.314249430754 (T) = (0 -37.12840737317261) / Math.Sqrt((0.01145128982304304 / (24)) + (0.010573844010070088 / (16))) is less than -2.0243941639098457 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (24) + (16) - 2, .025) and -1.6248887137268486 = (14.144754853419004 - 37.12840737317261) / 14.144754853419004 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Numerics.Tests.Perf_Quaternion.CreateFromRotationMatrixBenchmark
```log
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 17.74071543547166 > 16.523233185803527.
IsChangePoint: Marked as a change because one of 1/6/2023 8:18:17 PM, 2/2/2023 2:17:49 AM, 2/7/2023 2:48:42 AM falls between 1/29/2023 12:56:15 AM and 2/7/2023 2:48:42 AM.
IsRegressionStdDev: Marked as regression because -66.16114680045021 (T) = (0 -17.685975274609746) / Math.Sqrt((0.006977108161075092 / (24)) + (0.009905508520053573 / (16))) is less than -2.0243941639098457 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (24) + (16) - 2, .025) and -0.1271876354155651 = (15.69035599657672 - 17.685975274609746) / 15.69035599657672 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Numerics.Tests.Perf_Quaternion.DivisionOperatorBenchmark
```log
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 15.652727299059364 > 9.351297146848694.
IsChangePoint: Marked as a change because one of 2/2/2023 2:17:49 AM, 2/7/2023 2:48:42 AM falls between 1/29/2023 12:56:15 AM and 2/7/2023 2:48:42 AM.
IsRegressionStdDev: Marked as regression because -179.15302155289265 (T) = (0 -15.597904040419898) / Math.Sqrt((0.010926303958308299 / (24)) + (0.015376183196756387 / (16))) is less than -2.0243941639098457 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (24) + (16) - 2, .025) and -0.7613279646593847 = (8.85576357917891 - 15.597904040419898) / 8.85576357917891 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Numerics.Tests.Perf_Quaternion.ConcatenateBenchmark
```log
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 8.082194213649098 > 2.3080874927400212.
IsChangePoint: Marked as a change because one of 2/2/2023 2:17:49 AM, 2/7/2023 2:48:42 AM falls between 1/29/2023 12:56:15 AM and 2/7/2023 2:48:42 AM.
IsRegressionStdDev: Marked as regression because -158.75556827975439 (T) = (0 -8.044385993022145) / Math.Sqrt((0.005161299902007397 / (23)) + (0.018279518290545582 / (16))) is less than -2.026192463026769 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (16) - 2, .025) and -2.6985814767956176 = (2.174992235128926 - 8.044385993022145) / 2.174992235128926 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
Architecture | arm64 |
---|---|
OS | ubuntu 20.04 |
Baseline | 3c47b2c75dfca5a82a59db238f103cdbda902826 |
Compare | 873775b42431bf27a9ae69633864e5290443c29b |
Diff | Diff |
Regressions in System.Numerics.Tests.Perf_Matrix4x4
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
---|---|---|---|---|---|---|---|---|---|---|
TransformBenchmark - Duration of single invocation | 24.31 ns | 28.59 ns | 1.18 | 0.07 | False |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
Payloads
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Numerics.Tests.Perf_Matrix4x4*'
Payloads
Histogram
System.Numerics.Tests.Perf_Matrix4x4.TransformBenchmark
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 28.58503990700628 > 24.11130244544981.
IsChangePoint: Marked as a change because one of 2/2/2023 2:17:49 AM, 2/7/2023 2:48:42 AM falls between 1/29/2023 12:56:15 AM and 2/7/2023 2:48:42 AM.
IsRegressionStdDev: Marked as regression because -24.07581338992146 (T) = (0 -28.66686478868555) / Math.Sqrt((1.2286013034574401 / (23)) + (0.014949985175678191 / (16))) is less than -2.026192463026769 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (16) - 2, .025) and -0.24346838304287843 = (23.05395551637201 - 28.66686478868555) / 23.05395551637201 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
Architecture | arm64 |
---|---|
OS | ubuntu 20.04 |
Baseline | 3c47b2c75dfca5a82a59db238f103cdbda902826 |
Compare | 873775b42431bf27a9ae69633864e5290443c29b |
Diff | Diff |
Regressions in System.Numerics.Tests.Perf_Vector3
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
---|---|---|---|---|---|---|---|---|---|---|
TransformByQuaternionBenchmark - Duration of single invocation | 0.79 ns | 7.38 ns | 9.36 | 0.33 | False |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
Payloads
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Numerics.Tests.Perf_Vector3*'
Payloads
Histogram
System.Numerics.Tests.Perf_Vector3.TransformByQuaternionBenchmark
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 7.3838777325075755 > 0.8277504860603464.
IsChangePoint: Marked as a change because one of 1/22/2023 2:59:30 AM, 2/2/2023 2:17:49 AM, 2/7/2023 2:48:42 AM falls between 1/29/2023 12:56:15 AM and 2/7/2023 2:48:42 AM.
IsRegressionStdDev: Marked as regression because -126.99526607522674 (T) = (0 -7.35893009654001) / Math.Sqrt((0.05410990308401292 / (24)) + (0.00642522359262174 / (16))) is less than -2.0243941639098457 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (24) + (16) - 2, .025) and -8.042009880592124 = (0.8138599928247484 - 7.35893009654001) / 0.8138599928247484 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
Architecture | arm64 |
---|---|
OS | ubuntu 20.04 |
Baseline | 3c47b2c75dfca5a82a59db238f103cdbda902826 |
Compare | 873775b42431bf27a9ae69633864e5290443c29b |
Diff | Diff |
Regressions in System.Numerics.Tests.Perf_Vector2
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
---|---|---|---|---|---|---|---|---|---|---|
TransformByQuaternionBenchmark - Duration of single invocation | 0.04 ns | 2.46 ns | 55.28 | 0.78 | False |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
Payloads
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Numerics.Tests.Perf_Vector2*'
Payloads
Histogram
System.Numerics.Tests.Perf_Vector2.TransformByQuaternionBenchmark
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.4592848641998617 > 0.003390660399691646.
IsChangePoint: Marked as a change because one of 2/2/2023 2:17:49 AM, 2/7/2023 2:48:42 AM falls between 1/29/2023 12:56:15 AM and 2/7/2023 2:48:42 AM.
IsRegressionStdDev: Marked as regression because -27.399889821767825 (T) = (0 -2.5765840624988376) / Math.Sqrt((0.1773866162467119 / (24)) + (0.011222966552333794 / (16))) is less than -2.0243941639098457 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (24) + (16) - 2, .025) and -22.060985317555495 = (0.11172914023484405 - 2.5765840624988376) / 0.11172914023484405 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
Architecture | arm64 |
---|---|
OS | ubuntu 20.04 |
Baseline | 3c47b2c75dfca5a82a59db238f103cdbda902826 |
Compare | 873775b42431bf27a9ae69633864e5290443c29b |
Diff | Diff |
Regressions in System.Numerics.Tests.Perf_Vector4
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
---|---|---|---|---|---|---|---|---|---|---|
TransformByQuaternionBenchmark - Duration of single invocation | 1.34 ns | 8.04 ns | 6.02 | 0.29 | False | |||||
TransformVector2ByQuaternionBenchmark - Duration of single invocation | 0.49 ns | 5.80 ns | 11.93 | 0.40 | False | |||||
TransformVector3ByQuaternionBenchmark - Duration of single invocation | 1.30 ns | 8.04 ns | 6.17 | 0.26 | False |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
Payloads
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Numerics.Tests.Perf_Vector4*'
Payloads
Histogram
System.Numerics.Tests.Perf_Vector4.TransformByQuaternionBenchmark
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 8.038862144980559 > 1.375150017228637.
IsChangePoint: Marked as a change because one of 1/22/2023 2:59:30 AM, 2/2/2023 2:17:49 AM, 2/7/2023 2:48:42 AM falls between 1/29/2023 12:56:15 AM and 2/7/2023 2:48:42 AM.
IsRegressionStdDev: Marked as regression because -253.67002250969566 (T) = (0 -8.003021323808895) / Math.Sqrt((0.0031426920369748467 / (24)) + (0.00903622795642789 / (16))) is less than -2.0243941639098457 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (24) + (16) - 2, .025) and -5.09920332347962 = (1.3121420781301547 - 8.003021323808895) / 1.3121420781301547 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Numerics.Tests.Perf_Vector4.TransformVector2ByQuaternionBenchmark
```log
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 5.804749575425005 > 0.504518725267644.
IsChangePoint: Marked as a change because one of 12/28/2022 7:19:51 AM, 2/2/2023 2:17:49 AM, 2/7/2023 2:48:42 AM falls between 1/29/2023 12:56:15 AM and 2/7/2023 2:48:42 AM.
IsRegressionStdDev: Marked as regression because -168.98161044721124 (T) = (0 -5.964098047930127) / Math.Sqrt((0.011060216079014102 / (24)) + (0.009670498095240392 / (16))) is less than -2.0243941639098457 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (24) + (16) - 2, .025) and -12.28745338259838 = (0.44885185115613474 - 5.964098047930127) / 0.44885185115613474 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Numerics.Tests.Perf_Vector4.TransformVector3ByQuaternionBenchmark
```log
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 8.037643934677403 > 1.369175346531895.
IsChangePoint: Marked as a change because one of 1/22/2023 2:59:30 AM, 2/2/2023 2:17:49 AM, 2/7/2023 2:48:42 AM falls between 1/29/2023 12:56:15 AM and 2/7/2023 2:48:42 AM.
IsRegressionStdDev: Marked as regression because -196.70057792976218 (T) = (0 -7.9674953362402965) / Math.Sqrt((0.011146751982089 / (24)) + (0.01126782603079823 / (16))) is less than -2.0243941639098457 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (24) + (16) - 2, .025) and -5.409497985068502 = (1.2430763461976722 - 7.9674953362402965) / 1.2430763461976722 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository