[Perf] Windows/x64: 10 Improvements on 7/26/2023 8:42:30 AM #20266
Description
Run Information
| Name | Value |
|---|---|
| Architecture | x64 |
| OS | Windows 10.0.18362 |
| Queue | TigerWindows |
| Baseline | 8f5364695b107f37af899aba3ad1b9f98d577c33 |
| Compare | bc23bf4ee16cbd1999105b1916e1f2f76baef6a7 |
| Diff | Diff |
| Configs | CompilationMode:tiered, RunKind:micro |
Improvements in System.Collections.ContainsTrue<String>
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio |
|---|---|---|---|---|---|---|---|---|
| 463.14 μs | 379.19 μs | 0.82 | 0.23 | False | 5066414.141414141 | 3989126.0162601625 | 0.7873667459697076) | |
| 461.65 μs | 381.14 μs | 0.83 | 0.29 | False | 5062903.225806451 | 3989634.1463414636 | 0.7880131158750263) | |
| 461.79 μs | 377.77 μs | 0.82 | 0.28 | False | 5066935.483870967 | 3985670.7317073173 | 0.7866038050799099) | |
| 648.09 μs | 343.90 μs | 0.53 | 0.07 | False | 7404687.5 | 2238970.588235294 | 0.30237205665131633) | |
| 471.95 μs | 379.55 μs | 0.80 | 0.15 | False |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.ContainsTrue<String>*'Payloads
System.Collections.ContainsTrue<String>.Queue(Size: 512)
ETL Files
Histogram
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 379.18624530956856 < 440.2770168504902.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 7/26/2023 12:02:55 AM, 7/31/2023 7:57:50 PM falls between 7/23/2023 4:13:29 AM and 7/31/2023 7:57:50 PM.
IsImprovementStdDev: Marked as improvement because 54.08521456662321 (T) = (0 -379838.4545638461) / Math.Sqrt((1104557885.1574287 / (299)) + (2168996.3352966104 / (16))) is greater than 1.9675720194858106 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (16) - 2, .975) and 0.21792711236830745 = (485681.6552151419 - 379838.4545638461) / 485681.6552151419 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
System.Collections.ContainsTrue<String>.ICollection(Size: 512)
ETL Files
Histogram
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 381.13557926829264 < 440.9640379901961.
IsChangePoint: Marked as a change because one of 5/2/2023 3:18:41 PM, 5/8/2023 11:50:12 PM, 5/19/2023 10:30:38 AM, 7/26/2023 12:02:55 AM, 7/31/2023 7:57:50 PM falls between 7/23/2023 4:13:29 AM and 7/31/2023 7:57:50 PM.
IsImprovementStdDev: Marked as improvement because 47.53241643913536 (T) = (0 -380801.76421866426) / Math.Sqrt((1483882675.4461195 / (299)) + (1191543.497045877 / (17))) is greater than 1.9675476980854363 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (17) - 2, .975) and 0.21876670640795345 = (487436.6816444407 - 380801.76421866426) / 487436.6816444407 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
System.Collections.ContainsTrue<String>.List(Size: 512)
ETL Files
Histogram
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 377.77218324829937 < 442.27719761029414.
IsChangePoint: Marked as a change because one of 5/20/2023 3:57:43 PM, 7/26/2023 12:02:55 AM, 7/31/2023 7:57:50 PM falls between 7/23/2023 4:13:29 AM and 7/31/2023 7:57:50 PM.
IsImprovementStdDev: Marked as improvement because 33.11487983171403 (T) = (0 -381662.2185443965) / Math.Sqrt((1393587890.2252169 / (299)) + (86494969.59508859 / (16))) is greater than 1.9675720194858106 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (16) - 2, .975) and 0.21586405594579675 = (486729.6563030838 - 381662.2185443965) / 486729.6563030838 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
System.Collections.ContainsTrue<String>.LinkedList(Size: 512)
ETL Files
Histogram
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 343.89901315789473 < 615.149046875.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 7/26/2023 12:02:55 AM, 7/31/2023 7:57:50 PM falls between 7/23/2023 4:13:29 AM and 7/31/2023 7:57:50 PM.
IsImprovementStdDev: Marked as improvement because 24.18595273554422 (T) = (0 -368062.21930822765) / Math.Sqrt((1510628508.4353876 / (299)) + (2757558415.1423984 / (17))) is greater than 1.9675476980854363 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (17) - 2, .975) and 0.45941436347655695 = (680858.3033675669 - 368062.21930822765) / 680858.3033675669 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
System.Collections.ContainsTrue<String>.ImmutableArray(Size: 512)
ETL Files
Histogram
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 379.5477359693878 < 448.3893008578431.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 7/26/2023 12:02:55 AM, 7/31/2023 7:57:50 PM falls between 7/23/2023 4:13:29 AM and 7/31/2023 7:57:50 PM.
IsImprovementStdDev: Marked as improvement because 21.52385544446945 (T) = (0 -389996.47750167677) / Math.Sqrt((1501815668.1541963 / (299)) + (319874820.5637975 / (17))) is greater than 1.9675476980854363 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (17) - 2, .975) and 0.21226684574132004 = (495087.04234836315 - 389996.47750167677) / 495087.04234836315 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
| Name | Value |
|---|---|
| Architecture | x64 |
| OS | Windows 10.0.18362 |
| Queue | TigerWindows |
| Baseline | 8f5364695b107f37af899aba3ad1b9f98d577c33 |
| Compare | bc23bf4ee16cbd1999105b1916e1f2f76baef6a7 |
| Diff | Diff |
| Configs | CompilationMode:tiered, RunKind:micro |
Improvements in System.Text.Json.Serialization.Tests.WriteJson<ImmutableDictionary<String, String>>
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio |
|---|---|---|---|---|---|---|---|---|
| 22.86 μs | 17.14 μs | 0.75 | 0.08 | False | 187531.00198412698 | 151618.86724386725 | 0.8085002780324322) | |
| 22.37 μs | 16.79 μs | 0.75 | 0.10 | False | 186572.0524017467 | 201796.33143580012 | 1.081599997631322) | |
| 22.19 μs | 16.33 μs | 0.74 | 0.10 | False | 185523.80952380953 | 149365.07936507938 | 0.8050992470910336) | |
| 23.60 μs | 16.68 μs | 0.71 | 0.11 | False | 190977.39692001988 | 156007.10014947684 | 0.8168877713565843) |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Serialization.Tests.WriteJson<ImmutableDictionary<String, String>>*'Payloads
System.Text.Json.Serialization.Tests.WriteJson<ImmutableDictionary<String, String>>.SerializeToUtf8Bytes(Mode: Reflection)
ETL Files
Histogram
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 17.137926826877845 < 21.376327821225466.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 7/26/2023 12:02:55 AM, 7/31/2023 7:57:50 PM falls between 7/23/2023 4:13:29 AM and 7/31/2023 7:57:50 PM.
IsImprovementStdDev: Marked as improvement because 55.42622192814865 (T) = (0 -16427.130103253596) / Math.Sqrt((2768246.9959000223 / (299)) + (78524.91775260323 / (17))) is greater than 1.9675476980854363 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (17) - 2, .975) and 0.28442313773705646 = (22956.485836202646 - 16427.130103253596) / 22956.485836202646 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
System.Text.Json.Serialization.Tests.WriteJson<ImmutableDictionary<String, String>>.SerializeToStream(Mode: Reflection)
ETL Files
Histogram
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 16.786249104905117 < 21.24252505278783.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 7/26/2023 12:02:55 AM, 7/31/2023 7:57:50 PM falls between 7/23/2023 4:13:29 AM and 7/31/2023 7:57:50 PM.
IsImprovementStdDev: Marked as improvement because 34.91874767239423 (T) = (0 -16942.67374166452) / Math.Sqrt((2591709.309662356 / (299)) + (314560.78171418316 / (16))) is greater than 1.9675720194858106 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (16) - 2, .975) and 0.25754564942409314 = (22819.81879225629 - 16942.67374166452) / 22819.81879225629 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
System.Text.Json.Serialization.Tests.WriteJson<ImmutableDictionary<String, String>>.SerializeToWriter(Mode: Reflection)
ETL Files
Histogram
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 16.32830628272251 < 20.37777258814966.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 7/26/2023 12:02:55 AM, 7/31/2023 7:57:50 PM falls between 7/23/2023 4:13:29 AM and 7/31/2023 7:57:50 PM.
IsImprovementStdDev: Marked as improvement because 58.87935834141756 (T) = (0 -16396.521681883827) / Math.Sqrt((2576859.3491351795 / (299)) + (31004.915022518944 / (15))) is greater than 1.9675964973877862 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (15) - 2, .975) and 0.27070986631265237 = (22482.85137080045 - 16396.521681883827) / 22482.85137080045 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
System.Text.Json.Serialization.Tests.WriteJson<ImmutableDictionary<String, String>>.SerializeToString(Mode: Reflection)
ETL Files
Histogram
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 16.68181356837607 < 21.834303492966946.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 7/26/2023 12:02:55 AM, 7/31/2023 7:57:50 PM falls between 7/23/2023 4:13:29 AM and 7/31/2023 7:57:50 PM.
IsImprovementStdDev: Marked as improvement because 20.02457227583713 (T) = (0 -17575.669586644766) / Math.Sqrt((2706088.9831080446 / (299)) + (1373468.7364200342 / (17))) is greater than 1.9675476980854363 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (17) - 2, .975) and 0.25456672984549816 = (23577.790649191113 - 17575.669586644766) / 23577.790649191113 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
| Name | Value |
|---|---|
| Architecture | x64 |
| OS | Windows 10.0.18362 |
| Queue | TigerWindows |
| Baseline | 8f5364695b107f37af899aba3ad1b9f98d577c33 |
| Compare | bc23bf4ee16cbd1999105b1916e1f2f76baef6a7 |
| Diff | Diff |
| Configs | CompilationMode:tiered, RunKind:micro |
Improvements in System.Collections.ContainsFalse<String>
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio |
|---|---|---|---|---|---|---|---|---|
| 1.33 ms | 886.87 μs | 0.67 | 0.09 | False | 14494551.282051282 | 4175086.8055555555 | 0.2880452608923188) |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.ContainsFalse<String>*'Payloads
System.Collections.ContainsFalse<String>.LinkedList(Size: 512)
ETL Files
Histogram
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 886.8652213541666 < 1.2377633695913457.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 7/26/2023 12:02:55 AM, 7/31/2023 7:57:50 PM falls between 7/23/2023 4:13:29 AM and 7/31/2023 7:57:50 PM.
IsImprovementStdDev: Marked as improvement because 15.033593795949177 (T) = (0 -858355.288011696) / Math.Sqrt((5703088272.316532 / (299)) + (17765179534.143597 / (16))) is greater than 1.9675720194858106 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (16) - 2, .975) and 0.3705143259028219 = (1363581.9262174116 - 858355.288011696) / 1363581.9262174116 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository