- 
                Notifications
    You must be signed in to change notification settings 
- Fork 5.2k
Closed
Labels
arch-x86area-TieredCompilation-coreclrruntime-coreclrspecific to the CoreCLR runtimespecific to the CoreCLR runtimetenet-performancePerformance related issuePerformance related issuetenet-performance-benchmarksIssue from performance benchmarkIssue from performance benchmark
Milestone
Description
Run Information
| Architecture | x86 | 
|---|---|
| OS | Windows 10.0.18362 | 
| Baseline | daf25da8e67857fde603b16d92192cd1368bafc7 | 
| Compare | a29a33456bd4df703e4ff675f36fe0f22903953d | 
| Diff | Diff | 
Regressions in System.Collections.Tests.Perf_BitArray
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL | 
|---|---|---|---|---|---|---|---|---|---|---|
| BitArrayLengthCtor - Duration of single invocation | 11.66 ns | 13.55 ns | 1.16 | 0.18 | False | |||||
| BitArrayIntArrayCtor - Duration of single invocation | 144.79 ns | 153.79 ns | 1.06 | 0.00 | False | 
Repro
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.Tests.Perf_BitArray*'Payloads
Histogram
System.Collections.Tests.Perf_BitArray.BitArrayLengthCtor(Size: 512)
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 13.552456654774215 > 12.311515625815773.
IsChangePoint: Marked as a change because one of 4/10/2022 11:44:05 PM, 4/11/2022 5:01:30 PM, 4/12/2022 5:02:33 PM falls between 4/4/2022 4:14:58 AM and 4/12/2022 5:02:33 PM.
IsRegressionStdDev: Marked as regression because -3.1934874249260923 (T) = (0 -12.868695907343728) / Math.Sqrt((1.8689032056672892 / (53)) + (0.20798193329018388 / (4))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (53) + (4) - 2, .025) and -0.07910354378561829 = (11.925357841193696 - 12.868695907343728) / 11.925357841193696 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Collections.Tests.Perf_BitArray.BitArrayIntArrayCtor(Size: 512)
```log
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 153.78919173026836 > 152.37757986958002.
IsChangePoint: Marked as a change because one of 4/8/2022 10:19:59 AM, 4/12/2022 5:02:33 PM falls between 4/4/2022 4:14:58 AM and 4/12/2022 5:02:33 PM.
IsRegressionStdDev: Marked as regression because -22.810390319926345 (T) = (0 -153.674493616133) / Math.Sqrt((0.9275447022264903 / (37)) + (1.740421529380291 / (20))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (37) + (20) - 2, .025) and -0.05229391205141096 = (146.0376154001973 - 153.674493616133) / 146.0376154001973 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Metadata
Metadata
Assignees
Labels
arch-x86area-TieredCompilation-coreclrruntime-coreclrspecific to the CoreCLR runtimespecific to the CoreCLR runtimetenet-performancePerformance related issuePerformance related issuetenet-performance-benchmarksIssue from performance benchmarkIssue from performance benchmark

