[Perf] Linux/x64: 4 Regressions on 4/17/2023 11:29:15 PM

[performanceautofiler]

Run Information

Name	Value
Architecture	x64
OS	ubuntu 18.04
Queue	TigerUbuntu
Baseline	7dbf6a5f2fd636db350f9cc31b2dfd3af0eff1a8
Compare	eb71e484d2d4a7ae002ea524d2edead773289bbb
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Linq.Tests.Perf_Enumerable

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
SelectToArray - Duration of single invocation	359.26 ns	382.24 ns	1.06	0.28	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Linq.Tests.Perf_Enumerable*'

Payloads

Baseline
Compare

Histogram

System.Linq.Tests.Perf_Enumerable.SelectToArray(input: IList)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 382.24001019158584 > 377.330284047364.
IsChangePoint: Marked as a change because one of 4/17/2023 6:40:18 PM, 4/25/2023 3:46:39 AM falls between 4/12/2023 8:39:35 PM and 4/25/2023 3:46:39 AM.
IsRegressionStdDev: Marked as regression because -15.783998863328524 (T) = (0 -385.3218076900582) / Math.Sqrt((2.442310014790744 / (16)) + (85.31381222594095 / (33))) is less than -2.011740513728388 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (16) + (33) - 2, .025) and -0.07270844339487358 = (359.2046003391229 - 385.3218076900582) / 359.2046003391229 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	x64
OS	ubuntu 18.04
Queue	TigerUbuntu
Baseline	7dbf6a5f2fd636db350f9cc31b2dfd3af0eff1a8
Compare	eb71e484d2d4a7ae002ea524d2edead773289bbb
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Tests.Perf_Double

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
Parse - Duration of single invocation	47.85 ns	50.85 ns	1.06	0.01	False
TryParse - Duration of single invocation	47.49 ns	50.40 ns	1.06	0.01	False
Parse - Duration of single invocation	100.04 ns	107.22 ns	1.07	0.04	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Double*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_Double.Parse(value: "12345")

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 50.84558969377382 > 50.36091049699699.
IsChangePoint: Marked as a change because one of 2/9/2023 4:27:10 AM, 2/16/2023 9:36:08 AM, 4/17/2023 6:40:18 PM, 4/25/2023 3:46:39 AM falls between 4/12/2023 8:39:35 PM and 4/25/2023 3:46:39 AM.
IsRegressionStdDev: Marked as regression because -35.52684570706089 (T) = (0 -51.00878120755583) / Math.Sqrt((0.11875786643995419 / (16)) + (0.06035151492881923 / (33))) is less than -2.011740513728388 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (16) + (33) - 2, .025) and -0.07180001007884479 = (47.59169689110515 - 51.00878120755583) / 47.59169689110515 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Double.TryParse(value: "12345")

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 50.39988527160089 > 49.236540630552774.
IsChangePoint: Marked as a change because one of 2/9/2023 4:27:10 AM, 2/16/2023 9:36:08 AM, 4/17/2023 6:40:18 PM, 4/25/2023 3:46:39 AM falls between 4/12/2023 8:39:35 PM and 4/25/2023 3:46:39 AM.
IsRegressionStdDev: Marked as regression because -39.63685350536492 (T) = (0 -50.37582171541442) / Math.Sqrt((0.10284869771253022 / (16)) + (0.058905204353530104 / (33))) is less than -2.011740513728388 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (16) + (33) - 2, .025) and -0.07678154750384258 = (46.783697057396544 - 50.37582171541442) / 46.783697057396544 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Double.Parse(value: "1.7976931348623157e+308")

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 107.2234858395088 > 105.61062850281994.
IsChangePoint: Marked as a change because one of 2/9/2023 4:27:10 AM, 2/16/2023 9:36:08 AM, 3/3/2023 4:59:02 PM, 3/13/2023 2:14:21 PM, 4/17/2023 6:40:18 PM, 4/25/2023 3:46:39 AM falls between 4/12/2023 8:39:35 PM and 4/25/2023 3:46:39 AM.
IsRegressionStdDev: Marked as regression because -20.235812111324645 (T) = (0 -107.21317947826955) / Math.Sqrt((0.2984640894728244 / (15)) + (3.0107746128634396 / (33))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (15) + (33) - 2, .025) and -0.06714568124058198 = (100.46723831898163 - 107.21317947826955) / 100.46723831898163 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

[kunalspathak]

dotnet/runtime#84587