Closed
Description
Run Information
Architecture | x64 |
---|---|
OS | ubuntu 18.04 |
Baseline | bdd67af67dbf43c0f5e9151b98dc100b9555898a |
Compare | fbc20578253e0deb7267eea5a02b680c3cfe0e2a |
Diff | Diff |
Regressions in System.Numerics.Tests.Perf_VectorOf<UInt16>
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
---|---|---|---|---|---|---|---|---|---|---|
InequalityOperatorBenchmark - Duration of single invocation | 22.47 ns | 101.87 ns | 4.53 | 0.22 | False |
Repro
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Numerics.Tests.Perf_VectorOf<UInt16>*'
Related Issues
Regressions
- [Perf] Windows/x64: 28 Regressions on 11/2/2022 4:25:40 PM
- [Perf] Windows/x64: 31 Regressions on 11/2/2022 4:25:40 PM
- [Perf] Windows/x86: 30 Regressions on 11/2/2022 4:25:40 PM
- [Perf] Linux/x64: 15 Regressions on 11/2/2022 4:25:40 PM
- [Perf] Alpine/x64: 18 Regressions on 11/2/2022 4:25:40 PM
Improvements
- [Perf] Windows/x64: 5 Improvements on 11/2/2022 4:25:40 PM
- [Perf] Windows/x86: 3 Improvements on 11/2/2022 4:25:40 PM
Payloads
Histogram
Edge Detector Info
System.Numerics.Tests.Perf_VectorOf<UInt16>.InequalityOperatorBenchmark
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 101.8723735640621 > 23.646291113151594.
IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 11/8/2022 6:56:37 AM falls between 10/30/2022 5:36:44 PM and 11/8/2022 6:56:37 AM.
IsRegressionStdDev: Marked as regression because -79.25538140347159 (T) = (0 -103.4858080440732) / Math.Sqrt((0.37671569915901404 / (27)) + (20.514024812935062 / (20))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (27) + (20) - 2, .025) and -3.5640094338646824 = (22.67431948676849 - 103.4858080440732) / 22.67431948676849 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Architecture | x64 |
---|---|
OS | ubuntu 18.04 |
Baseline | bdd67af67dbf43c0f5e9151b98dc100b9555898a |
Compare | 803688595257517f4b47e873d11152b0560c204a |
Diff | Diff |
Regressions in System.Numerics.Tests.Perf_VectorOf<Double>
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
---|---|---|---|---|---|---|---|---|---|---|
BitwiseAndOperatorBenchmark - Duration of single invocation | 17.05 ns | 40.95 ns | 2.40 | 0.26 | False |
Repro
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Numerics.Tests.Perf_VectorOf<Double>*'
Related Issues
Regressions
- [Perf] Windows/x64: 28 Regressions on 11/2/2022 4:25:40 PM
- [Perf] Windows/x64: 31 Regressions on 11/2/2022 4:25:40 PM
- [Perf] Windows/x86: 30 Regressions on 11/2/2022 4:25:40 PM
- [Perf] Linux/x64: 15 Regressions on 11/2/2022 4:25:40 PM
- [Perf] Alpine/x64: 18 Regressions on 11/2/2022 4:25:40 PM
Improvements
- [Perf] Windows/x64: 5 Improvements on 11/2/2022 4:25:40 PM
- [Perf] Windows/x86: 3 Improvements on 11/2/2022 4:25:40 PM
Payloads
Histogram
Edge Detector Info
System.Numerics.Tests.Perf_VectorOf<Double>.BitwiseAndOperatorBenchmark
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 40.95460802841224 > 17.89092744701803.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 11/2/2022 9:35:40 AM, 11/8/2022 6:56:37 AM falls between 10/30/2022 5:36:44 PM and 11/8/2022 6:56:37 AM.
IsRegressionStdDev: Marked as regression because -61.7586337497908 (T) = (0 -40.92899037459878) / Math.Sqrt((3.550268126415335 / (27)) + (0.12886468606461166 / (15))) is less than -2.0210753903043583 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (27) + (15) - 2, .025) and -1.2975451308767367 = (17.814226943598882 - 40.92899037459878) / 17.814226943598882 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
Architecture | x64 |
---|---|
OS | ubuntu 18.04 |
Baseline | bdd67af67dbf43c0f5e9151b98dc100b9555898a |
Compare | fbc20578253e0deb7267eea5a02b680c3cfe0e2a |
Diff | Diff |
Regressions in System.Numerics.Tests.Perf_VectorOf<Int16>
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
---|---|---|---|---|---|---|---|---|---|---|
BitwiseAndOperatorBenchmark - Duration of single invocation | 17.24 ns | 46.23 ns | 2.68 | 0.26 | False | |||||
BitwiseOrOperatorBenchmark - Duration of single invocation | 16.83 ns | 41.12 ns | 2.44 | 0.23 | False | |||||
BitwiseAndBenchmark - Duration of single invocation | 17.63 ns | 41.01 ns | 2.33 | 0.24 | False |
Repro
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Numerics.Tests.Perf_VectorOf<Int16>*'
Related Issues
Regressions
- [Perf] Windows/x64: 28 Regressions on 11/2/2022 4:25:40 PM
- [Perf] Windows/x64: 31 Regressions on 11/2/2022 4:25:40 PM
- [Perf] Windows/x86: 30 Regressions on 11/2/2022 4:25:40 PM
- [Perf] Linux/x64: 15 Regressions on 11/2/2022 4:25:40 PM
- [Perf] Alpine/x64: 18 Regressions on 11/2/2022 4:25:40 PM
Improvements
- [Perf] Windows/x64: 5 Improvements on 11/2/2022 4:25:40 PM
- [Perf] Windows/x86: 3 Improvements on 11/2/2022 4:25:40 PM
Payloads
Histogram
Edge Detector Info
System.Numerics.Tests.Perf_VectorOf<Int16>.BitwiseAndOperatorBenchmark
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 46.23171662618531 > 19.204274366632728.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 11/2/2022 9:35:40 AM, 11/8/2022 6:56:37 AM falls between 10/30/2022 5:36:44 PM and 11/8/2022 6:56:37 AM.
IsRegressionStdDev: Marked as regression because -54.56970294733505 (T) = (0 -41.62094914874507) / Math.Sqrt((1.6947133397427672 / (27)) + (2.631937690725204 / (20))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (27) + (20) - 2, .025) and -1.3698157546097423 = (17.56294727460749 - 41.62094914874507) / 17.56294727460749 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Numerics.Tests.Perf_VectorOf<Int16>.BitwiseOrOperatorBenchmark
```log
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 41.123642253570885 > 17.67390587361091.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 11/2/2022 9:35:40 AM, 11/8/2022 6:56:37 AM falls between 10/30/2022 5:36:44 PM and 11/8/2022 6:56:37 AM.
IsRegressionStdDev: Marked as regression because -46.65548334193179 (T) = (0 -42.2698202917497) / Math.Sqrt((0.725433647671959 / (27)) + (5.146797230008012 / (20))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (27) + (20) - 2, .025) and -1.4296840421516723 = (17.39724983101775 - 42.2698202917497) / 17.39724983101775 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Numerics.Tests.Perf_VectorOf<Int16>.BitwiseAndBenchmark
```log
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 41.00786437152644 > 18.106941145872657.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 11/2/2022 9:35:40 AM, 11/8/2022 6:56:37 AM falls between 10/30/2022 5:36:44 PM and 11/8/2022 6:56:37 AM.
IsRegressionStdDev: Marked as regression because -47.971942304492586 (T) = (0 -41.89848386482377) / Math.Sqrt((0.9583090918739371 / (27)) + (3.369308689495923 / (15))) is less than -2.0210753903043583 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (27) + (15) - 2, .025) and -1.4035167613112252 = (17.432157969211033 - 41.89848386482377) / 17.432157969211033 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
Architecture | x64 |
---|---|
OS | ubuntu 18.04 |
Baseline | bdd67af67dbf43c0f5e9151b98dc100b9555898a |
Compare | 803688595257517f4b47e873d11152b0560c204a |
Diff | Diff |
Regressions in System.Numerics.Tests.Perf_VectorConvert
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
---|---|---|---|---|---|---|---|---|---|---|
Narrow_ulong - Duration of single invocation | 79.78 μs | 104.95 μs | 1.32 | 0.20 | False | |||||
Widen_float - Duration of single invocation | 145.57 μs | 205.33 μs | 1.41 | 0.17 | False | |||||
Convert_int_float - Duration of single invocation | 73.89 μs | 97.92 μs | 1.33 | 0.24 | False | |||||
Convert_double_ulong - Duration of single invocation | 52.04 μs | 81.86 μs | 1.57 | 0.19 | False | |||||
Convert_double_long - Duration of single invocation | 49.22 μs | 74.94 μs | 1.52 | 0.22 | False | |||||
Narrow_long - Duration of single invocation | 79.00 μs | 103.93 μs | 1.32 | 0.23 | False |
Repro
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Numerics.Tests.Perf_VectorConvert*'
Related Issues
Regressions
- [Perf] Windows/x64: 28 Regressions on 11/2/2022 4:25:40 PM
- [Perf] Windows/x64: 31 Regressions on 11/2/2022 4:25:40 PM
- [Perf] Windows/x86: 30 Regressions on 11/2/2022 4:25:40 PM
- [Perf] Linux/x64: 15 Regressions on 11/2/2022 4:25:40 PM
- [Perf] Alpine/x64: 18 Regressions on 11/2/2022 4:25:40 PM
Improvements
- [Perf] Windows/x64: 4 Improvements on 11/2/2022 4:25:40 PM
- [Perf] Windows/x64: 5 Improvements on 11/2/2022 4:25:40 PM
- [Perf] Windows/x86: 3 Improvements on 11/2/2022 4:25:40 PM
Payloads
Histogram
Edge Detector Info
System.Numerics.Tests.Perf_VectorConvert.Narrow_ulong
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 104.95298112416106 > 83.82619170245998.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 11/2/2022 9:35:40 AM, 11/8/2022 6:56:37 AM falls between 10/30/2022 5:36:44 PM and 11/8/2022 6:56:37 AM.
IsRegressionStdDev: Marked as regression because -15.028197043380729 (T) = (0 -108498.67190792745) / Math.Sqrt((20640245.424279176 / (27)) + (41394914.23234916 / (15))) is less than -2.0210753903043583 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (27) + (15) - 2, .025) and -0.3513886502803742 = (80286.80119920876 - 108498.67190792745) / 80286.80119920876 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Numerics.Tests.Perf_VectorConvert.Widen_float
```log
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 205.33210570175436 > 152.89814630497685.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 11/2/2022 9:35:40 AM, 11/8/2022 6:56:37 AM falls between 10/30/2022 5:36:44 PM and 11/8/2022 6:56:37 AM.
IsRegressionStdDev: Marked as regression because -46.03248274675961 (T) = (0 -206791.55442326108) / Math.Sqrt((24541186.451466978 / (27)) + (12172409.53414969 / (15))) is less than -2.0210753903043583 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (27) + (15) - 2, .025) and -0.41238491623099965 = (146413.0295125863 - 206791.55442326108) / 146413.0295125863 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Numerics.Tests.Perf_VectorConvert.Convert_int_float
```log
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 97.92457967355038 > 78.65790162056324.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/10/2022 9:53:50 PM, 11/2/2022 9:35:40 AM, 11/8/2022 6:56:37 AM falls between 10/30/2022 5:36:44 PM and 11/8/2022 6:56:37 AM.
IsRegressionStdDev: Marked as regression because -29.31764783983678 (T) = (0 -99345.42528445156) / Math.Sqrt((8551349.57005463 / (27)) + (9892849.651633168 / (20))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (27) + (20) - 2, .025) and -0.3620639825262765 = (72937.41451131502 - 99345.42528445156) / 72937.41451131502 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Numerics.Tests.Perf_VectorConvert.Convert_double_ulong
```log
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 81.86362716721594 > 54.66164615240883.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 11/2/2022 9:35:40 AM, 11/8/2022 6:56:37 AM falls between 10/30/2022 5:36:44 PM and 11/8/2022 6:56:37 AM.
IsRegressionStdDev: Marked as regression because -40.13389904845121 (T) = (0 -79254.30966647006) / Math.Sqrt((2485774.038005878 / (27)) + (7364011.244383776 / (20))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (27) + (20) - 2, .025) and -0.5233509330800098 = (52026.29804166565 - 79254.30966647006) / 52026.29804166565 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Numerics.Tests.Perf_VectorConvert.Convert_double_long
```log
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 74.93954069199322 > 51.69727490201422.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 11/2/2022 9:35:40 AM, 11/8/2022 6:56:37 AM falls between 10/30/2022 5:36:44 PM and 11/8/2022 6:56:37 AM.
IsRegressionStdDev: Marked as regression because -31.010454455633266 (T) = (0 -76429.69267691964) / Math.Sqrt((8255383.496955883 / (27)) + (8760225.391375972 / (20))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (27) + (20) - 2, .025) and -0.5382635855833126 = (49685.69326689051 - 76429.69267691964) / 49685.69326689051 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Numerics.Tests.Perf_VectorConvert.Narrow_long
```log
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 103.933983995585 > 82.88653661375476.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 11/2/2022 9:35:40 AM, 11/8/2022 6:56:37 AM falls between 10/30/2022 5:36:44 PM and 11/8/2022 6:56:37 AM.
IsRegressionStdDev: Marked as regression because -29.327984135105677 (T) = (0 -104380.06520311843) / Math.Sqrt((16518238.505915344 / (27)) + (1416676.1425518903 / (15))) is less than -2.0210753903043583 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (27) + (15) - 2, .025) and -0.309111242962731 = (79733.53354363485 - 104380.06520311843) / 79733.53354363485 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
Architecture | x64 |
---|---|
OS | ubuntu 18.04 |
Baseline | bdd67af67dbf43c0f5e9151b98dc100b9555898a |
Compare | 803688595257517f4b47e873d11152b0560c204a |
Diff | Diff |
Regressions in System.Numerics.Tests.Perf_VectorOf<UInt32>
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
---|---|---|---|---|---|---|---|---|---|---|
BitwiseOrOperatorBenchmark - Duration of single invocation | 17.01 ns | 41.21 ns | 2.42 | 0.25 | False | |||||
BitwiseAndOperatorBenchmark - Duration of single invocation | 17.71 ns | 41.00 ns | 2.32 | 0.19 | False |
Repro
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Numerics.Tests.Perf_VectorOf<UInt32>*'
Related Issues
Regressions
- [Perf] Windows/x64: 28 Regressions on 11/2/2022 4:25:40 PM
- [Perf] Windows/x64: 31 Regressions on 11/2/2022 4:25:40 PM
- [Perf] Windows/x86: 30 Regressions on 11/2/2022 4:25:40 PM
- [Perf] Linux/x64: 15 Regressions on 11/2/2022 4:25:40 PM
- [Perf] Alpine/x64: 18 Regressions on 11/2/2022 4:25:40 PM
Improvements
- [Perf] Windows/x64: 4 Improvements on 11/2/2022 4:25:40 PM
- [Perf] Windows/x64: 5 Improvements on 11/2/2022 4:25:40 PM
- [Perf] Windows/x86: 3 Improvements on 11/2/2022 4:25:40 PM
Payloads
Histogram
Edge Detector Info
System.Numerics.Tests.Perf_VectorOf<UInt32>.BitwiseOrOperatorBenchmark
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 41.209746789214364 > 17.66995864271164.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 11/2/2022 9:35:40 AM, 11/8/2022 6:56:37 AM falls between 10/30/2022 5:36:44 PM and 11/8/2022 6:56:37 AM.
IsRegressionStdDev: Marked as regression because -89.54246276073859 (T) = (0 -41.62412908651522) / Math.Sqrt((0.26413727330486136 / (27)) + (0.9815957020861871 / (15))) is less than -2.0210753903043583 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (27) + (15) - 2, .025) and -1.4390725814118643 = (17.065555737755442 - 41.62412908651522) / 17.065555737755442 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Numerics.Tests.Perf_VectorOf<UInt32>.BitwiseAndOperatorBenchmark
```log
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 41.00427073607222 > 18.17271721873155.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 11/2/2022 9:35:40 AM, 11/8/2022 6:56:37 AM falls between 10/30/2022 5:36:44 PM and 11/8/2022 6:56:37 AM.
IsRegressionStdDev: Marked as regression because -60.442393532133 (T) = (0 -41.659199935449834) / Math.Sqrt((0.6286674176296809 / (27)) + (2.087449203957097 / (15))) is less than -2.0210753903043583 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (27) + (15) - 2, .025) and -1.4083181882940397 = (17.298046469914183 - 41.659199935449834) / 17.298046469914183 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
Architecture | x64 |
---|---|
OS | ubuntu 18.04 |
Baseline | bdd67af67dbf43c0f5e9151b98dc100b9555898a |
Compare | 803688595257517f4b47e873d11152b0560c204a |
Diff | Diff |
Regressions in System.Numerics.Tests.Perf_VectorOf<UInt64>
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
---|---|---|---|---|---|---|---|---|---|---|
BitwiseAndBenchmark - Duration of single invocation | 17.40 ns | 41.48 ns | 2.38 | 0.32 | False | |||||
AndNotBenchmark - Duration of single invocation | 47.23 ns | 68.53 ns | 1.45 | 0.21 | False | |||||
BitwiseOrBenchmark - Duration of single invocation | 16.99 ns | 41.29 ns | 2.43 | 0.27 | False | |||||
BitwiseOrOperatorBenchmark - Duration of single invocation | 16.68 ns | 41.04 ns | 2.46 | 0.25 | False |
Repro
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Numerics.Tests.Perf_VectorOf<UInt64>*'
Related Issues
Regressions
- [Perf] Windows/x64: 28 Regressions on 11/2/2022 4:25:40 PM
- [Perf] Windows/x64: 31 Regressions on 11/2/2022 4:25:40 PM
- [Perf] Windows/x86: 30 Regressions on 11/2/2022 4:25:40 PM
- [Perf] Linux/x64: 15 Regressions on 11/2/2022 4:25:40 PM
- [Perf] Alpine/x64: 18 Regressions on 11/2/2022 4:25:40 PM
Improvements
- [Perf] Windows/x64: 4 Improvements on 11/2/2022 4:25:40 PM
- [Perf] Windows/x64: 5 Improvements on 11/2/2022 4:25:40 PM
- [Perf] Windows/x86: 3 Improvements on 11/2/2022 4:25:40 PM
Payloads
Histogram
Edge Detector Info
System.Numerics.Tests.Perf_VectorOf<UInt64>.BitwiseAndBenchmark
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 41.48398040340836 > 18.007254320146036.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 11/2/2022 9:35:40 AM, 11/8/2022 6:56:37 AM falls between 10/30/2022 5:36:44 PM and 11/8/2022 6:56:37 AM.
IsRegressionStdDev: Marked as regression because -38.72305734344954 (T) = (0 -41.85410487326282) / Math.Sqrt((0.7372721120516357 / (27)) + (5.779947676987692 / (15))) is less than -2.0210753903043583 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (27) + (15) - 2, .025) and -1.4649547091155404 = (16.979664867059828 - 41.85410487326282) / 16.979664867059828 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Numerics.Tests.Perf_VectorOf<UInt64>.AndNotBenchmark
```log
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 68.53079839824615 > 49.59741285334486.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 11/2/2022 9:35:40 AM, 11/8/2022 6:56:37 AM falls between 10/30/2022 5:36:44 PM and 11/8/2022 6:56:37 AM.
IsRegressionStdDev: Marked as regression because -26.743445228124298 (T) = (0 -69.569958025041) / Math.Sqrt((6.959244070546707 / (27)) + (6.191331620439273 / (15))) is less than -2.0210753903043583 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (27) + (15) - 2, .025) and -0.4593691978649538 = (47.6712528445998 - 69.569958025041) / 47.6712528445998 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Numerics.Tests.Perf_VectorOf<UInt64>.BitwiseOrBenchmark
```log
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 41.29102544643759 > 17.837997867879324.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 11/2/2022 9:35:40 AM, 11/8/2022 6:56:37 AM falls between 10/30/2022 5:36:44 PM and 11/8/2022 6:56:37 AM.
IsRegressionStdDev: Marked as regression because -84.32194503404915 (T) = (0 -41.57919965089723) / Math.Sqrt((0.4316546353520266 / (27)) + (1.024167009304162 / (15))) is less than -2.0210753903043583 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (27) + (15) - 2, .025) and -1.4312679953394747 = (17.101857849731445 - 41.57919965089723) / 17.101857849731445 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Numerics.Tests.Perf_VectorOf<UInt64>.BitwiseOrOperatorBenchmark
```log
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 41.04113880156161 > 17.57886486632404.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 11/2/2022 9:35:40 AM, 11/8/2022 6:56:37 AM falls between 10/30/2022 5:36:44 PM and 11/8/2022 6:56:37 AM.
IsRegressionStdDev: Marked as regression because -113.10015922039341 (T) = (0 -41.3385978615966) / Math.Sqrt((0.46826771432638165 / (27)) + (0.5724168770163124 / (20))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (27) + (20) - 2, .025) and -1.418765228745505 = (17.09078556707916 - 41.3385978615966) / 17.09078556707916 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
Architecture | x64 |
---|---|
OS | ubuntu 18.04 |
Baseline | bdd67af67dbf43c0f5e9151b98dc100b9555898a |
Compare | 803688595257517f4b47e873d11152b0560c204a |
Diff | Diff |
Regressions in System.Numerics.Tests.Perf_VectorOf<Single>
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
---|---|---|---|---|---|---|---|---|---|---|
InequalityOperatorBenchmark - Duration of single invocation | 26.20 ns | 66.18 ns | 2.53 | 0.18 | False | |||||
ExclusiveOrOperatorBenchmark - Duration of single invocation | 17.08 ns | 41.70 ns | 2.44 | 0.31 | False |
Repro
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Numerics.Tests.Perf_VectorOf<Single>*'
Related Issues
Regressions
- [Perf] Windows/x64: 28 Regressions on 11/2/2022 4:25:40 PM
- [Perf] Windows/x64: 31 Regressions on 11/2/2022 4:25:40 PM
- [Perf] Windows/x86: 30 Regressions on 11/2/2022 4:25:40 PM
- [Perf] Linux/x64: 15 Regressions on 11/2/2022 4:25:40 PM
- [Perf] Alpine/x64: 18 Regressions on 11/2/2022 4:25:40 PM
Improvements
- [Perf] Windows/x64: 4 Improvements on 11/2/2022 4:25:40 PM
- [Perf] Windows/x64: 5 Improvements on 11/2/2022 4:25:40 PM
- [Perf] Windows/x86: 3 Improvements on 11/2/2022 4:25:40 PM
Payloads
Histogram
Edge Detector Info
System.Numerics.Tests.Perf_VectorOf<Single>.InequalityOperatorBenchmark
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 66.18122802510553 > 25.716330206758997.
IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 11/8/2022 6:56:37 AM falls between 10/30/2022 5:36:44 PM and 11/8/2022 6:56:37 AM.
IsRegressionStdDev: Marked as regression because -72.04483808295696 (T) = (0 -66.92945269364074) / Math.Sqrt((0.8037325946287306 / (27)) + (4.77826178437499 / (15))) is less than -2.0210753903043583 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (27) + (15) - 2, .025) and -1.74192593849726 = (24.409650076224192 - 66.92945269364074) / 24.409650076224192 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Numerics.Tests.Perf_VectorOf<Single>.ExclusiveOrOperatorBenchmark
```log
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 41.70045337304935 > 17.94259740037124.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 11/2/2022 9:35:40 AM, 11/8/2022 6:56:37 AM falls between 10/30/2022 5:36:44 PM and 11/8/2022 6:56:37 AM.
IsRegressionStdDev: Marked as regression because -115.66960144342421 (T) = (0 -41.31350116025097) / Math.Sqrt((0.506024790701155 / (27)) + (0.365550294310876 / (15))) is less than -2.0210753903043583 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (27) + (15) - 2, .025) and -1.3885310462048988 = (17.296614681183808 - 41.31350116025097) / 17.296614681183808 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository