-
Notifications
You must be signed in to change notification settings - Fork 4
Open
Description
Run Information
Name | Value |
---|---|
Architecture | x64 |
OS | ubuntu 18.04 |
Queue | TigerUbuntu |
Baseline | 33b18692fdf2ef438de3da38bc419c6616a746c5 |
Compare | 7222061f18b8342a0bd4edbd37df318fd43c47d3 |
Diff | Diff |
Configs | AOT:true, CompilationMode:wasm, RunKind:micro |
Improvements in System.Tests.Perf_Uri
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
---|---|---|---|---|---|---|---|---|---|---|
Ctor - Duration of single invocation | 410.41 ns | 364.89 ns | 0.89 | 0.08 | False | |||||
Ctor - Duration of single invocation | 1.25 μs | 1.06 μs | 0.84 | 0.27 | False |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
Payloads
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Uri*'
Payloads
Histogram
System.Tests.Perf_Uri.Ctor(input: "https://a.much.longer.domain.name")
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 364.89463643789446 < 388.9025260008287.
IsChangePoint: Marked as a change because one of 3/2/2023 11:15:04 AM, 4/19/2023 8:38:27 PM, 4/25/2023 3:46:39 AM falls between 4/12/2023 8:39:35 PM and 4/25/2023 3:46:39 AM.
IsImprovementStdDev: Marked as improvement because 16.933554056252213 (T) = (0 -369.5412789040005) / Math.Sqrt((166.54410642862229 / (299)) + (46.006277151429636 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.06827500803490213 = (396.6205501524675 - 369.5412789040005) / 396.6205501524675 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
System.Tests.Perf_Uri.Ctor(input: "http://höst.with.ünicode")
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.0593218531341109 < 1.1614960796346543.
IsChangePoint: Marked as a change because one of 3/11/2023 1:35:17 AM, 4/12/2023 8:39:35 PM, 4/25/2023 3:46:39 AM falls between 4/12/2023 8:39:35 PM and 4/25/2023 3:46:39 AM.
IsImprovementStdDev: Marked as improvement because 22.1725352857483 (T) = (0 -1115.4114076652997) / Math.Sqrt((2821.474482401777 / (299)) + (1275.239985697304 / (35))) is greater than 1.9671350567190904 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (35) - 2, .975) and 0.11865814343699432 = (1265.5831552300283 - 1115.4114076652997) / 1265.5831552300283 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
Name | Value |
---|---|
Architecture | x64 |
OS | ubuntu 18.04 |
Queue | TigerUbuntu |
Baseline | 33b18692fdf2ef438de3da38bc419c6616a746c5 |
Compare | 7222061f18b8342a0bd4edbd37df318fd43c47d3 |
Diff | Diff |
Configs | AOT:true, CompilationMode:wasm, RunKind:micro |
Improvements in System.Tests.Perf_String
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
---|---|---|---|---|---|---|---|---|---|---|
ToLowerInvariant - Duration of single invocation | 278.63 ns | 214.46 ns | 0.77 | 0.18 | False | |||||
Concat_str_str - Duration of single invocation | 530.52 ns | 477.29 ns | 0.90 | 0.03 | False | |||||
ToLower - Duration of single invocation | 320.30 ns | 274.97 ns | 0.86 | 0.23 | False | |||||
ToUpperInvariant - Duration of single invocation | 265.13 ns | 222.61 ns | 0.84 | 0.22 | False |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
Payloads
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_String*'
Payloads
Histogram
System.Tests.Perf_String.ToLowerInvariant(s: "This is a much longer piece of text that might benefit more from vectorization.")
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 214.45963642370026 < 276.3277209058668.
IsChangePoint: Marked as a change because one of 3/1/2023 3:09:21 PM, 4/19/2023 8:38:27 PM, 4/25/2023 3:46:39 AM falls between 4/12/2023 8:39:35 PM and 4/25/2023 3:46:39 AM.
IsImprovementStdDev: Marked as improvement because 9.044792719323247 (T) = (0 -219.43259829069942) / Math.Sqrt((2431.116231480251 / (299)) + (535.3116094325152 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.1876475979139465 = (270.119959917906 - 219.43259829069942) / 270.119959917906 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
System.Tests.Perf_String.Concat_str_str(size: 1000)
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 477.29353310678613 < 497.5502179047296.
IsChangePoint: Marked as a change because one of 2/14/2023 11:21:41 AM, 3/10/2023 8:06:53 PM, 4/12/2023 8:39:35 PM, 4/25/2023 3:46:39 AM falls between 4/12/2023 8:39:35 PM and 4/25/2023 3:46:39 AM.
IsImprovementStdDev: Marked as improvement because 30.175451051270922 (T) = (0 -480.50584399348145) / Math.Sqrt((952.4446921327024 / (299)) + (46.39243047034044 / (37))) is greater than 1.9670919629190173 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (37) - 2, .975) and 0.11685410607929513 = (544.0843322729921 - 480.50584399348145) / 544.0843322729921 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
System.Tests.Perf_String.ToLower(s: "This is a much longer piece of text that might benefit more from vectorization.")
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 274.9684342540415 < 317.8845530456082.
IsChangePoint: Marked as a change because one of 3/1/2023 3:09:21 PM, 4/19/2023 8:38:27 PM, 4/25/2023 3:46:39 AM falls between 4/12/2023 8:39:35 PM and 4/25/2023 3:46:39 AM.
IsImprovementStdDev: Marked as improvement because 8.029164269045607 (T) = (0 -272.07402373560404) / Math.Sqrt((2994.762040272043 / (299)) + (222.91770603186137 / (22))) is greater than 1.9674283869023508 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (22) - 2, .975) and 0.11697145702200322 = (308.11464238521626 - 272.07402373560404) / 308.11464238521626 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
System.Tests.Perf_String.ToUpperInvariant(s: "This is a much longer piece of text that might benefit more from vectorization.")
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 222.6112856440589 < 254.213208486426.
IsChangePoint: Marked as a change because one of 3/1/2023 3:09:21 PM, 4/19/2023 8:38:27 PM, 4/25/2023 3:46:39 AM falls between 4/12/2023 8:39:35 PM and 4/25/2023 3:46:39 AM.
IsImprovementStdDev: Marked as improvement because 15.159819568783325 (T) = (0 -216.05769560995213) / Math.Sqrt((2562.596638620794 / (299)) + (91.95993219266492 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.1992022751567803 = (269.8030837341002 - 216.05769560995213) / 269.8030837341002 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
Name | Value |
---|---|
Architecture | x64 |
OS | ubuntu 18.04 |
Queue | TigerUbuntu |
Baseline | 33b18692fdf2ef438de3da38bc419c6616a746c5 |
Compare | 7222061f18b8342a0bd4edbd37df318fd43c47d3 |
Diff | Diff |
Configs | AOT:true, CompilationMode:wasm, RunKind:micro |
Improvements in System.Tests.Perf_DateTime
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
---|---|---|---|---|---|---|---|---|---|---|
ToString - Duration of single invocation | 487.85 ns | 162.65 ns | 0.33 | 0.13 | False |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
Payloads
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_DateTime*'
Payloads
Histogram
System.Tests.Perf_DateTime.ToString(format: "s")
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 162.6512292283251 < 463.8037888581452.
IsChangePoint: Marked as a change because one of 4/9/2023 4:09:45 AM, 4/11/2023 10:53:42 PM, 4/19/2023 8:38:27 PM, 4/25/2023 3:46:39 AM falls between 4/12/2023 8:39:35 PM and 4/25/2023 3:46:39 AM.
IsImprovementStdDev: Marked as improvement because 85.43025550893505 (T) = (0 -159.2572811785712) / Math.Sqrt((1937.088128457099 / (299)) + (18.08483844104 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.5911456201290549 = (389.520790333323 - 159.2572811785712) / 389.520790333323 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository