Closed
Description
Run Information
Architecture | x64 |
---|---|
OS | ubuntu 18.04 |
Baseline | 8c58fc2347820ce48e09605d8adddb993df9ebb5 |
Compare | 1d15f2140f7eb30a976c66290491ec89cd628da0 |
Diff | Diff |
Improvements in System.Numerics.Tests.Perf_Matrix4x4
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
---|---|---|---|---|---|---|---|---|---|---|
CreateBillboardBenchmark - Duration of single invocation | 17.35 ns | 14.95 ns | 0.86 | 0.05 | False | |||||
CreateConstrainedBillboardBenchmark - Duration of single invocation | 22.54 ns | 21.03 ns | 0.93 | 0.03 | False | |||||
CreateLookAtBenchmark - Duration of single invocation | 23.86 ns | 21.27 ns | 0.89 | 0.03 | False |
Repro
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Numerics.Tests.Perf_Matrix4x4*'
Payloads
Histogram
System.Numerics.Tests.Perf_Matrix4x4.CreateBillboardBenchmark
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 14.949935649858014 < 16.53165531925393.
IsChangePoint: Marked as a change because one of 12/28/2022 5:13:50 PM, 1/3/2023 6:29:42 AM falls between 12/23/2022 9:40:21 PM and 1/3/2023 6:29:42 AM.
IsImprovementStdDev: Marked as improvement because 48.56187748338224 (T) = (0 -15.17351207429175) / Math.Sqrt((0.08356905932027467 / (299)) + (0.016326580865739154 / (14))) is greater than 1.9676211333067681 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (14) - 2, .975) and 0.10848596460596345 = (17.019936279057315 - 15.17351207429175) / 17.019936279057315 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Numerics.Tests.Perf_Matrix4x4.CreateConstrainedBillboardBenchmark
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 21.028890185157586 < 21.566710749759952.
IsChangePoint: Marked as a change because one of 10/31/2022 1:09:40 PM, 12/28/2022 5:13:50 PM, 1/3/2023 6:29:42 AM falls between 12/23/2022 9:40:21 PM and 1/3/2023 6:29:42 AM.
IsImprovementStdDev: Marked as improvement because 34.3427353145053 (T) = (0 -21.06048486161879) / Math.Sqrt((0.1784762407458816 / (299)) + (0.02696755398750096 / (14))) is greater than 1.9676211333067681 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (14) - 2, .975) and 0.07570908096091228 = (22.785558559326443 - 21.06048486161879) / 22.785558559326443 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Numerics.Tests.Perf_Matrix4x4.CreateLookAtBenchmark
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 21.265391138740927 < 22.742165856061746.
IsChangePoint: Marked as a change because one of 10/10/2022 3:36:43 PM, 12/28/2022 5:13:50 PM, 1/3/2023 6:29:42 AM falls between 12/23/2022 9:40:21 PM and 1/3/2023 6:29:42 AM.
IsImprovementStdDev: Marked as improvement because 36.05286792903522 (T) = (0 -21.5773947106395) / Math.Sqrt((0.13816616229161455 / (299)) + (0.05185477059484998 / (14))) is greater than 1.9676211333067681 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (14) - 2, .975) and 0.09734680460824881 = (23.904412924916215 - 21.5773947106395) / 23.904412924916215 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Architecture | x64 |
---|---|
OS | ubuntu 18.04 |
Baseline | 8c58fc2347820ce48e09605d8adddb993df9ebb5 |
Compare | 1d15f2140f7eb30a976c66290491ec89cd628da0 |
Diff | Diff |
Improvements in System.Text.Encodings.Web.Tests.Perf_Encoders
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
---|---|---|---|---|---|---|---|---|---|---|
EncodeUtf8 - Duration of single invocation | 43.09 ns | 35.92 ns | 0.83 | 0.06 | False |
Repro
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Encodings.Web.Tests.Perf_Encoders*'
Payloads
Histogram
System.Text.Encodings.Web.Tests.Perf_Encoders.EncodeUtf8(arguments: JavaScript,&Hello+<World>!,16)
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 35.92314244080981 < 40.78805184681181.
IsChangePoint: Marked as a change because one of 11/7/2022 10:11:20 AM, 11/10/2022 9:13:35 AM, 12/1/2022 7:29:59 PM, 12/7/2022 8:33:47 PM, 12/17/2022 10:44:50 PM, 12/21/2022 9:23:17 AM, 12/28/2022 5:13:50 PM, 1/3/2023 6:29:42 AM falls between 12/23/2022 9:40:21 PM and 1/3/2023 6:29:42 AM.
IsImprovementStdDev: Marked as improvement because 35.43333133398268 (T) = (0 -35.92910143654136) / Math.Sqrt((8.154406749203927 / (299)) + (0.0015330160028067831 / (14))) is greater than 1.9676211333067681 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (14) - 2, .975) and 0.14029600095459205 = (41.79240933674388 - 35.92910143654136) / 41.79240933674388 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
Architecture | x64 |
---|---|
OS | ubuntu 18.04 |
Baseline | 8c58fc2347820ce48e09605d8adddb993df9ebb5 |
Compare | 1d15f2140f7eb30a976c66290491ec89cd628da0 |
Diff | Diff |
Improvements in System.Collections.IterateFor<Int32>
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
---|---|---|---|---|---|---|---|---|---|---|
ImmutableList - Duration of single invocation | 12.04 μs | 11.13 μs | 0.93 | 0.04 | False |
Repro
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.IterateFor<Int32>*'
Payloads
Histogram
System.Collections.IterateFor<Int32>.ImmutableList(Size: 512)
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 11.134748633274231 < 11.313731682449374.
IsChangePoint: Marked as a change because one of 11/3/2022 3:03:31 PM, 11/7/2022 4:10:19 PM, 11/18/2022 7:31:41 AM, 11/18/2022 8:48:12 PM, 12/1/2022 7:29:59 PM, 12/6/2022 2:12:43 AM, 12/28/2022 5:13:50 PM, 1/3/2023 6:29:42 AM falls between 12/23/2022 9:40:21 PM and 1/3/2023 6:29:42 AM.
IsImprovementStdDev: Marked as improvement because 27.368660755867264 (T) = (0 -11143.890740674022) / Math.Sqrt((146307.08177978286 / (299)) + (913.1473280075089 / (14))) is greater than 1.9676211333067681 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (14) - 2, .975) and 0.054672370043298064 = (11788.38995871139 - 11143.890740674022) / 11788.38995871139 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment