Description
Run Information
Architecture | x64 |
---|---|
OS | alpine 3.15 |
Baseline | bdd67af67dbf43c0f5e9151b98dc100b9555898a |
Compare | 803688595257517f4b47e873d11152b0560c204a |
Diff | Diff |
Regressions in System.Numerics.Tests.Perf_VectorOf<Single>
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
---|---|---|---|---|---|---|---|---|---|---|
OnesComplementOperatorBenchmark - Duration of single invocation | 0.03 ns | 5.15 ns | 189.46 | 0.13 | False | |||||
OnesComplementBenchmark - Duration of single invocation | 0.20 ns | 5.15 ns | 25.33 | 0.12 | False |
Repro
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Numerics.Tests.Perf_VectorOf<Single>*'
Related Issues
Regressions
Improvements
Payloads
Histogram
Edge Detector Info
System.Numerics.Tests.Perf_VectorOf<Single>.OnesComplementOperatorBenchmark
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 5.147585974035645 > 0.028505902855850897.
IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 11/8/2022 2:29:58 AM falls between 10/30/2022 12:44:54 AM and 11/8/2022 2:29:58 AM.
IsRegressionStdDev: Marked as regression because -17.035723730410535 (T) = (0 -5.313293826900229) / Math.Sqrt((0.008671072049271684 / (31)) + (1.3078897861220542 / (14))) is less than -2.016692199226234 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (31) + (14) - 2, .025) and -52.90391568533566 = (0.09856971908899169 - 5.313293826900229) / 0.09856971908899169 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Numerics.Tests.Perf_VectorOf<Single>.OnesComplementBenchmark
```log
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 5.147095774848735 > 0.21320171866483373.
IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 11/8/2022 2:29:58 AM falls between 10/30/2022 12:44:54 AM and 11/8/2022 2:29:58 AM.
IsRegressionStdDev: Marked as regression because -154.30012216448975 (T) = (0 -5.002563886124615) / Math.Sqrt((0.008144905190398946 / (31)) + (0.010163029318325061 / (14))) is less than -2.016692199226234 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (31) + (14) - 2, .025) and -32.15431838940555 = (0.15088724875499723 - 5.002563886124615) / 0.15088724875499723 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Architecture | x64 |
---|---|
OS | alpine 3.15 |
Baseline | bdd67af67dbf43c0f5e9151b98dc100b9555898a |
Compare | fbc20578253e0deb7267eea5a02b680c3cfe0e2a |
Diff | Diff |
Regressions in System.Numerics.Tests.Perf_VectorConvert
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
---|---|---|---|---|---|---|---|---|---|---|
Convert_float_uint - Duration of single invocation | 5.97 μs | 8.67 μs | 1.45 | 0.13 | False | |||||
Convert_double_long - Duration of single invocation | 4.94 μs | 5.28 μs | 1.07 | 0.03 | False |
Repro
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Numerics.Tests.Perf_VectorConvert*'
Related Issues
Regressions
Improvements
Payloads
Histogram
Edge Detector Info
System.Numerics.Tests.Perf_VectorConvert.Convert_float_uint
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 8.669842122083734 > 6.183840251113515.
IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 11/8/2022 2:29:58 AM falls between 10/30/2022 12:44:54 AM and 11/8/2022 2:29:58 AM.
IsRegressionStdDev: Marked as regression because -92.30254166812752 (T) = (0 -8709.511423521046) / Math.Sqrt((17309.338845812174 / (31)) + (8157.529313894346 / (19))) is less than -2.010634757623041 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (31) + (19) - 2, .025) and -0.499405918628244 = (5808.641486148784 - 8709.511423521046) / 5808.641486148784 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Numerics.Tests.Perf_VectorConvert.Convert_double_long
```log
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 5.278643359375 > 5.1956581855486474.
IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 11/8/2022 2:29:58 AM falls between 10/30/2022 12:44:54 AM and 11/8/2022 2:29:58 AM.
IsRegressionStdDev: Marked as regression because -10.001170908270584 (T) = (0 -5352.4019895686915) / Math.Sqrt((9852.90359011977 / (31)) + (11778.00435724762 / (19))) is less than -2.010634757623041 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (31) + (19) - 2, .025) and -0.060691852625918696 = (5046.14226678364 - 5352.4019895686915) / 5046.14226678364 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
Architecture | x64 |
---|---|
OS | alpine 3.15 |
Baseline | bdd67af67dbf43c0f5e9151b98dc100b9555898a |
Compare | fbc20578253e0deb7267eea5a02b680c3cfe0e2a |
Diff | Diff |
Regressions in System.Numerics.Tests.Perf_VectorOf<Int32>
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
---|---|---|---|---|---|---|---|---|---|---|
OnesComplementBenchmark - Duration of single invocation | 0.20 ns | 5.15 ns | 25.25 | 0.10 | False | |||||
OnesComplementOperatorBenchmark - Duration of single invocation | 0.03 ns | 4.93 ns | 184.61 | 0.17 | False | |||||
SquareRootBenchmark - Duration of single invocation | 12.73 ns | 14.81 ns | 1.16 | 0.12 | False |
Repro
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Numerics.Tests.Perf_VectorOf<Int32>*'
Related Issues
Regressions
Improvements
Payloads
Histogram
Edge Detector Info
System.Numerics.Tests.Perf_VectorOf<Int32>.OnesComplementBenchmark
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 5.146855526337509 > 0.21339541745701465.
IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 11/8/2022 2:29:58 AM falls between 10/30/2022 12:44:54 AM and 11/8/2022 2:29:58 AM.
IsRegressionStdDev: Marked as regression because -166.4479425627722 (T) = (0 -5.043758845473797) / Math.Sqrt((0.007180352852090967 / (31)) + (0.01207942149522186 / (19))) is less than -2.010634757623041 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (31) + (19) - 2, .025) and -34.610065274364395 = (0.1416385734374036 - 5.043758845473797) / 0.1416385734374036 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Numerics.Tests.Perf_VectorOf<Int32>.OnesComplementOperatorBenchmark
```log
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 4.929505157251311 > 0.028009220260416585.
IsChangePoint: Marked as a change because one of 11/2/2022 4:14:20 AM, 11/8/2022 2:29:58 AM falls between 10/30/2022 12:44:54 AM and 11/8/2022 2:29:58 AM.
IsRegressionStdDev: Marked as regression because -30.6283865804121 (T) = (0 -4.9745268329382055) / Math.Sqrt((0.7035059514692314 / (30)) + (0.006170778654317372 / (20))) is less than -2.010634757623041 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (30) + (20) - 2, .025) and -18.622815430445506 = (0.2535072936180222 - 4.9745268329382055) / 0.2535072936180222 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Numerics.Tests.Perf_VectorOf<Int32>.SquareRootBenchmark
```log
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 14.813095486527583 > 13.367106407771924.
IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 11/8/2022 2:29:58 AM falls between 10/30/2022 12:44:54 AM and 11/8/2022 2:29:58 AM.
IsRegressionStdDev: Marked as regression because -2.6462122208563033 (T) = (0 -15.053696972193919) / Math.Sqrt((8.531051482676718 / (31)) + (0.02238614410886596 / (19))) is less than -2.010634757623041 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (31) + (19) - 2, .025) and -0.101821860432917 = (13.662550647052118 - 15.053696972193919) / 13.662550647052118 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
Architecture | x64 |
---|---|
OS | alpine 3.15 |
Baseline | bdd67af67dbf43c0f5e9151b98dc100b9555898a |
Compare | fbc20578253e0deb7267eea5a02b680c3cfe0e2a |
Diff | Diff |
Regressions in System.Numerics.Tests.Perf_VectorOf<Double>
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
---|---|---|---|---|---|---|---|---|---|---|
OnesComplementBenchmark - Duration of single invocation | 0.20 ns | 5.13 ns | 25.17 | 0.09 | False | |||||
OnesComplementOperatorBenchmark - Duration of single invocation | 0.03 ns | 4.93 ns | 182.00 | 0.12 | False |
Repro
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Numerics.Tests.Perf_VectorOf<Double>*'
Related Issues
Regressions
Improvements
Payloads
Histogram
Edge Detector Info
System.Numerics.Tests.Perf_VectorOf<Double>.OnesComplementBenchmark
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 5.132380929997292 > 0.2135979984536395.
IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 11/8/2022 2:29:58 AM falls between 10/30/2022 12:44:54 AM and 11/8/2022 2:29:58 AM.
IsRegressionStdDev: Marked as regression because -31.446573344163472 (T) = (0 -5.000665118485749) / Math.Sqrt((0.6752374212738903 / (31)) + (0.012060293862679858 / (19))) is less than -2.010634757623041 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (31) + (19) - 2, .025) and -16.100447934087537 = (0.29242889646870407 - 5.000665118485749) / 0.29242889646870407 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Numerics.Tests.Perf_VectorOf<Double>.OnesComplementOperatorBenchmark
```log
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 4.930758162136195 > 0.028042913050541803.
IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 11/8/2022 2:29:58 AM falls between 10/30/2022 12:44:54 AM and 11/8/2022 2:29:58 AM.
IsRegressionStdDev: Marked as regression because -17.41900979855766 (T) = (0 -5.257846980347648) / Math.Sqrt((0.5320853843583266 / (31)) + (1.2620842108499604 / (19))) is less than -2.010634757623041 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (31) + (19) - 2, .025) and -22.71729350019317 = (0.2216883212371945 - 5.257846980347648) / 0.2216883212371945 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
Architecture | x64 |
---|---|
OS | alpine 3.15 |
Baseline | bdd67af67dbf43c0f5e9151b98dc100b9555898a |
Compare | fbc20578253e0deb7267eea5a02b680c3cfe0e2a |
Diff | Diff |
Regressions in System.Numerics.Tests.Perf_VectorOf<SByte>
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
---|---|---|---|---|---|---|---|---|---|---|
OnesComplementOperatorBenchmark - Duration of single invocation | 0.03 ns | 5.11 ns | 188.16 | 0.17 | False | |||||
MultiplyBenchmark - Duration of single invocation | 42.39 ns | 46.61 ns | 1.10 | 0.05 | False | |||||
OnesComplementBenchmark - Duration of single invocation | 0.20 ns | 4.93 ns | 24.24 | 0.16 | False |
Repro
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Numerics.Tests.Perf_VectorOf<SByte>*'
Related Issues
Regressions
Improvements
Payloads
Histogram
Edge Detector Info
System.Numerics.Tests.Perf_VectorOf<SByte>.OnesComplementOperatorBenchmark
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 5.108925273779534 > 0.02784170177431546.
IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 11/8/2022 2:29:58 AM falls between 10/30/2022 12:44:54 AM and 11/8/2022 2:29:58 AM.
IsRegressionStdDev: Marked as regression because -11.234651588335128 (T) = (0 -5.659785016763671) / Math.Sqrt((0.46358177711037346 / (31)) + (4.190229250176063 / (19))) is less than -2.010634757623041 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (31) + (19) - 2, .025) and -26.226195599999123 = (0.2078801276504402 - 5.659785016763671) / 0.2078801276504402 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Numerics.Tests.Perf_VectorOf<SByte>.MultiplyBenchmark
```log
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 46.611017325037466 > 44.55271377182026.
IsChangePoint: Marked as a change because one of 7/28/2022 3:42:13 AM, 9/22/2022 3:16:12 AM, 11/1/2022 2:33:16 AM, 11/8/2022 2:29:58 AM falls between 10/30/2022 12:44:54 AM and 11/8/2022 2:29:58 AM.
IsRegressionStdDev: Marked as regression because -39.41120700917367 (T) = (0 -46.26001104963115) / Math.Sqrt((0.005019493180415514 / (22)) + (0.25533311347249793 / (28))) is less than -2.010634757623041 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (22) + (28) - 2, .025) and -0.08976062086596563 = (42.449699653187295 - 46.26001104963115) / 42.449699653187295 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Numerics.Tests.Perf_VectorOf<SByte>.OnesComplementBenchmark
```log
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 4.927959637327005 > 0.21347979482183235.
IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 11/8/2022 2:29:58 AM falls between 10/30/2022 12:44:54 AM and 11/8/2022 2:29:58 AM.
IsRegressionStdDev: Marked as regression because -12.222975301824937 (T) = (0 -5.592413917242185) / Math.Sqrt((0.5112312129166333 / (31)) + (3.295594611455707 / (19))) is less than -2.010634757623041 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (31) + (19) - 2, .025) and -20.077023282078162 = (0.26533224556417445 - 5.592413917242185) / 0.26533224556417445 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
Architecture | x64 |
---|---|
OS | alpine 3.15 |
Baseline | bdd67af67dbf43c0f5e9151b98dc100b9555898a |
Compare | fbc20578253e0deb7267eea5a02b680c3cfe0e2a |
Diff | Diff |
Regressions in System.Numerics.Tests.Perf_VectorOf<UInt16>
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
---|---|---|---|---|---|---|---|---|---|---|
OnesComplementOperatorBenchmark - Duration of single invocation | 0.03 ns | 5.15 ns | 190.10 | 0.13 | False | |||||
OnesComplementBenchmark - Duration of single invocation | 0.20 ns | 4.93 ns | 24.27 | 0.10 | False |
Repro
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Numerics.Tests.Perf_VectorOf<UInt16>*'
Related Issues
Regressions
Improvements
Payloads
Histogram
Edge Detector Info
System.Numerics.Tests.Perf_VectorOf<UInt16>.OnesComplementOperatorBenchmark
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 5.146572204727077 > 0.02840506023073687.
IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 11/8/2022 2:29:58 AM falls between 10/30/2022 12:44:54 AM and 11/8/2022 2:29:58 AM.
IsRegressionStdDev: Marked as regression because -15.449168948829767 (T) = (0 -5.054156446854283) / Math.Sqrt((2.682698688909199 / (31)) + (0.012832351433235457 / (19))) is less than -2.010634757623041 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (31) + (19) - 2, .025) and -9.278861832269063 = (0.4917038996465016 - 5.054156446854283) / 0.4917038996465016 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Numerics.Tests.Perf_VectorOf<UInt16>.OnesComplementBenchmark
```log
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 4.930746071673132 > 0.21337762250773182.
IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 11/8/2022 2:29:58 AM falls between 10/30/2022 12:44:54 AM and 11/8/2022 2:29:58 AM.
IsRegressionStdDev: Marked as regression because -17.077558885402382 (T) = (0 -4.974446974362556) / Math.Sqrt((2.2043209386497957 / (31)) + (0.006283841984696833 / (19))) is less than -2.010634757623041 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (31) + (19) - 2, .025) and -11.13336668935149 = (0.40998076640412107 - 4.974446974362556) / 0.40998076640412107 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
Architecture | x64 |
---|---|
OS | alpine 3.15 |
Baseline | bdd67af67dbf43c0f5e9151b98dc100b9555898a |
Compare | 803688595257517f4b47e873d11152b0560c204a |
Diff | Diff |
Regressions in System.Numerics.Tests.Perf_VectorOf<Int64>
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
---|---|---|---|---|---|---|---|---|---|---|
OnesComplementBenchmark - Duration of single invocation | 0.20 ns | 4.93 ns | 24.26 | 0.13 | False |
Repro
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Numerics.Tests.Perf_VectorOf<Int64>*'
Related Issues
Regressions
Improvements
Payloads
Histogram
Edge Detector Info
System.Numerics.Tests.Perf_VectorOf<Int64>.OnesComplementBenchmark
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 4.932176625029846 > 0.21351500876185228.
IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 11/8/2022 2:29:58 AM falls between 10/30/2022 12:44:54 AM and 11/8/2022 2:29:58 AM.
IsRegressionStdDev: Marked as regression because -36.08536290746997 (T) = (0 -4.971610708407903) / Math.Sqrt((0.5093062784331596 / (31)) + (0.006435595777626207 / (14))) is less than -2.016692199226234 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (31) + (14) - 2, .025) and -16.62660326997716 = (0.2820515463053447 - 4.971610708407903) / 0.2820515463053447 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
Architecture | x64 |
---|---|
OS | alpine 3.15 |
Baseline | bdd67af67dbf43c0f5e9151b98dc100b9555898a |
Compare | fbc20578253e0deb7267eea5a02b680c3cfe0e2a |
Diff | Diff |
Regressions in System.Numerics.Tests.Perf_VectorOf<UInt32>
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
---|---|---|---|---|---|---|---|---|---|---|
OnesComplementOperatorBenchmark - Duration of single invocation | 0.03 ns | 4.93 ns | 176.60 | 0.09 | False | |||||
SquareRootBenchmark - Duration of single invocation | 13.16 ns | 15.44 ns | 1.17 | 0.10 | False | |||||
OnesComplementBenchmark - Duration of single invocation | 0.20 ns | 4.93 ns | 24.20 | 0.14 | False |
Repro
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Numerics.Tests.Perf_VectorOf<UInt32>*'
Related Issues
Regressions
Improvements
Payloads
Histogram
Edge Detector Info
System.Numerics.Tests.Perf_VectorOf<UInt32>.OnesComplementOperatorBenchmark
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 4.92800533628846 > 0.028365548029987114.
IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 11/8/2022 2:29:58 AM falls between 10/30/2022 12:44:54 AM and 11/8/2022 2:29:58 AM.
IsRegressionStdDev: Marked as regression because -36.22961533095226 (T) = (0 -4.975393292685176) / Math.Sqrt((0.5241031688175755 / (31)) + (0.006803184214938107 / (19))) is less than -2.010634757623041 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (31) + (19) - 2, .025) and -22.140599583067182 = (0.21500710363295222 - 4.975393292685176) / 0.21500710363295222 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Numerics.Tests.Perf_VectorOf<UInt32>.SquareRootBenchmark
```log
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 15.439441216595945 > 13.822807797463616.
IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 11/8/2022 2:29:58 AM falls between 10/30/2022 12:44:54 AM and 11/8/2022 2:29:58 AM.
IsRegressionStdDev: Marked as regression because -4.264698435137 (T) = (0 -15.996745039677549) / Math.Sqrt((4.590571669413593 / (31)) + (3.327598636700953 / (19))) is less than -2.010634757623041 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (31) + (19) - 2, .025) and -0.17864392679114122 = (13.57216091820767 - 15.996745039677549) / 13.57216091820767 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Numerics.Tests.Perf_VectorOf<UInt32>.OnesComplementBenchmark
```log
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 4.931015028187513 > 0.21383891876739705.
IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 11/8/2022 2:29:58 AM falls between 10/30/2022 12:44:54 AM and 11/8/2022 2:29:58 AM.
IsRegressionStdDev: Marked as regression because -175.92957837556838 (T) = (0 -5.00732477802344) / Math.Sqrt((0.0075898777848040265 / (31)) + (0.009917337242134457 / (19))) is less than -2.010634757623041 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (31) + (19) - 2, .025) and -35.91842730016665 = (0.13563212585712817 - 5.00732477802344) / 0.13563212585712817 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository