[Perf] Windows/arm64: 28 Regressions on 7/18/2023 1:13:00 AM #89259

performanceautofiler · 2023-07-20T08:20:39Z

Run Information

Name	Value
Architecture	arm64
OS	Windows 10.0.19041
Queue	SurfaceWindows
Baseline	bb38848e9877dc0b43860dc219ebe36fc04a6bb3
Compare	16dc5330051a97aec4ba89d21263cdccc7b7552c
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Text.Perf_Utf8Encoding

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector
GetBytes - Duration of single invocation 📝 - Benchmark Source	134.64 μs	163.41 μs	1.21	0.03	False
GetBytes - Duration of single invocation 📝 - Benchmark Source	144.34 μs	190.21 μs	1.32	0.01	False
GetString - Duration of single invocation 📝 - Benchmark Source	319.91 μs	361.24 μs	1.13	0.00	False
GetString - Duration of single invocation 📝 - Benchmark Source	200.01 μs	236.03 μs	1.18	0.00	False
GetBytes - Duration of single invocation 📝 - Benchmark Source	159.97 μs	208.18 μs	1.30	0.02	False
GetString - Duration of single invocation 📝 - Benchmark Source	187.68 μs	251.38 μs	1.34	0.05	False
GetBytes - Duration of single invocation 📝 - Benchmark Source	203.39 μs	238.89 μs	1.17	0.02	False
GetString - Duration of single invocation 📝 - Benchmark Source	44.30 μs	135.22 μs	3.05	0.01	False
GetBytes - Duration of single invocation 📝 - Benchmark Source	42.38 μs	78.88 μs	1.86	0.04	False
GetString - Duration of single invocation 📝 - Benchmark Source	261.72 μs	306.76 μs	1.17	0.01	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Perf_Utf8Encoding*'

Payloads

Baseline
Compare

System.Text.Perf_Utf8Encoding.GetBytes(Input: Cyrillic)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 163.41458333333333 > 141.77467636148876.
IsChangePoint: Marked as a change because one of 3/6/2023 6:03:21 AM, 4/4/2023 10:11:10 PM, 5/19/2023 4:22:12 AM, 7/17/2023 4:07:27 PM, 7/19/2023 6:59:29 PM falls between 7/10/2023 2:17:11 PM and 7/19/2023 6:59:29 PM.
IsRegressionStdDev: Marked as regression because -156.9943601397984 (T) = (0 -163313.67105263157) / Math.Sqrt((332399.08210140927 / (15)) + (20366.576792853983 / (2))) is less than -2.131449545559758 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (15) + (2) - 2, .025) and -0.20901931770273424 = (135079.45544074927 - 163313.67105263157) / 135079.45544074927 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Perf_Utf8Encoding.GetBytes(Input: EnglishMostlyAscii)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 190.21237931910568 > 151.46738830222137.
IsChangePoint: Marked as a change because one of 5/6/2023 11:12:53 AM, 7/17/2023 4:07:27 PM, 7/19/2023 6:59:29 PM falls between 7/10/2023 2:17:11 PM and 7/19/2023 6:59:29 PM.
IsRegressionStdDev: Marked as regression because -57.93111328415174 (T) = (0 -191002.3032508227) / Math.Sqrt((264013.2822441723 / (15)) + (1247959.6357985316 / (2))) is less than -2.131449545559758 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (15) + (2) - 2, .025) and -0.32089916063131374 = (144600.21547711076 - 191002.3032508227) / 144600.21547711076 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Perf_Utf8Encoding.GetString(Input: Greek)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 361.24470930232553 > 335.2484196428572.
IsChangePoint: Marked as a change because one of 3/16/2023 8:14:39 PM, 4/18/2023 4:36:42 PM, 6/22/2023 12:06:43 AM, 7/17/2023 4:07:27 PM, 7/19/2023 6:59:29 PM falls between 7/10/2023 2:17:11 PM and 7/19/2023 6:59:29 PM.
IsRegressionStdDev: Marked as regression because -14.118172520751896 (T) = (0 -364439.96422182466) / Math.Sqrt((156432.7000792867 / (15)) + (20419308.001166407 / (2))) is less than -2.131449545559758 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (15) + (2) - 2, .025) and -0.14135101329335703 = (319305.770071765 - 364439.96422182466) / 319305.770071765 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Perf_Utf8Encoding.GetString(Input: Cyrillic)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 236.02786796536802 > 209.01690668706294.
IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 6/22/2023 12:06:43 AM, 7/17/2023 4:07:27 PM, 7/19/2023 6:59:29 PM falls between 7/10/2023 2:17:11 PM and 7/19/2023 6:59:29 PM.
IsRegressionStdDev: Marked as regression because -31.45956270973919 (T) = (0 -237201.51176948054) / Math.Sqrt((948388.27929041 / (15)) + (2754879.557863424 / (2))) is less than -2.131449545559758 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (15) + (2) - 2, .025) and -0.18932988111238652 = (199441.31189878515 - 237201.51176948054) / 199441.31189878515 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Perf_Utf8Encoding.GetBytes(Input: Chinese)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 208.17916666666665 > 167.74164674908423.
IsChangePoint: Marked as a change because one of 5/6/2023 11:12:53 AM, 5/19/2023 4:22:12 AM, 6/22/2023 12:06:43 AM, 7/17/2023 4:07:27 PM, 7/19/2023 6:59:29 PM falls between 7/10/2023 2:17:11 PM and 7/19/2023 6:59:29 PM.
IsRegressionStdDev: Marked as regression because -105.10239953388513 (T) = (0 -207929.19060106558) / Math.Sqrt((2181557.5854358426 / (15)) + (124976.06674677305 / (2))) is less than -2.131449545559758 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (15) + (2) - 2, .025) and -0.2995270615430891 = (160003.7404024242 - 207929.19060106558) / 160003.7404024242 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Perf_Utf8Encoding.GetString(Input: EnglishMostlyAscii)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 251.37670546558707 > 199.26802149662032.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 7/17/2023 4:07:27 PM, 7/19/2023 6:59:29 PM falls between 7/10/2023 2:17:11 PM and 7/19/2023 6:59:29 PM.
IsRegressionStdDev: Marked as regression because -49.931345545245065 (T) = (0 -252501.08962795482) / Math.Sqrt((3578319.5423276178 / (15)) + (2528479.4891668903 / (2))) is less than -2.131449545559758 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (15) + (2) - 2, .025) and -0.31998419269850836 = (191290.99501695886 - 252501.08962795482) / 191290.99501695886 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Perf_Utf8Encoding.GetBytes(Input: Greek)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 238.88602810650892 > 213.3333234577922.
IsChangePoint: Marked as a change because one of 3/6/2023 6:03:21 AM, 3/14/2023 8:42:09 PM, 3/21/2023 2:53:21 PM, 4/4/2023 10:11:10 PM, 5/19/2023 4:22:12 AM, 7/17/2023 4:07:27 PM, 7/19/2023 6:59:29 PM falls between 7/10/2023 2:17:11 PM and 7/19/2023 6:59:29 PM.
IsRegressionStdDev: Marked as regression because -46.44101491427244 (T) = (0 -238147.19893087685) / Math.Sqrt((337750.0174463117 / (15)) + (1091737.1015303673 / (2))) is less than -2.131449545559758 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (15) + (2) - 2, .025) and -0.17236098584785833 = (203134.70151742332 - 238147.19893087685) / 203134.70151742332 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Perf_Utf8Encoding.GetString(Input: EnglishAllAscii)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 135.2222883064516 > 46.67656657750623.
IsChangePoint: Marked as a change because one of 7/17/2023 4:07:27 PM, 7/19/2023 6:59:29 PM falls between 7/10/2023 2:17:11 PM and 7/19/2023 6:59:29 PM.
IsRegressionStdDev: Marked as regression because -10.519272418769509 (T) = (0 -144750.35992703532) / Math.Sqrt((105388.31582920565 / (15)) + (181568297.61394644 / (2))) is less than -2.131449545559758 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (15) + (2) - 2, .025) and -2.2514945147506236 = (44518.10060584927 - 144750.35992703532) / 44518.10060584927 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Perf_Utf8Encoding.GetBytes(Input: EnglishAllAscii)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 78.88215829439251 > 44.312702125.
IsChangePoint: Marked as a change because one of 4/11/2023 5:52:15 AM, 7/17/2023 4:07:27 PM, 7/19/2023 6:59:29 PM falls between 7/10/2023 2:17:11 PM and 7/19/2023 6:59:29 PM.
IsRegressionStdDev: Marked as regression because -20.02921160723858 (T) = (0 -77139.65478125423) / Math.Sqrt((224515.86856747864 / (15)) + (6072636.986598566 / (2))) is less than -2.131449545559758 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (15) + (2) - 2, .025) and -0.8300022207196299 = (42152.765667639374 - 77139.65478125423) / 42152.765667639374 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Perf_Utf8Encoding.GetString(Input: Chinese)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 306.76269403594773 > 275.3442643065268.
IsChangePoint: Marked as a change because one of 3/16/2023 8:14:39 PM, 4/25/2023 6:46:56 PM, 7/17/2023 4:07:27 PM, 7/19/2023 6:59:29 PM falls between 7/10/2023 2:17:11 PM and 7/19/2023 6:59:29 PM.
IsRegressionStdDev: Marked as regression because -108.94988157600964 (T) = (0 -306379.07110760966) / Math.Sqrt((241114.1247489303 / (15)) + (294333.10229330295 / (2))) is less than -2.131449545559758 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (15) + (2) - 2, .025) and -0.16778130939409266 = (262359.9715485903 - 306379.07110760966) / 262359.9715485903 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name	Value
Architecture	arm64
OS	Windows 10.0.19041
Queue	SurfaceWindows
Baseline	bb38848e9877dc0b43860dc219ebe36fc04a6bb3
Compare	16dc5330051a97aec4ba89d21263cdccc7b7552c
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Text.Tests.Perf_StringBuilder

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector
ToString_MultipleSegments - Duration of single invocation 📝 - Benchmark Source	21.13 μs	96.64 μs	4.57	0.07	False
ToString_SingleSegment - Duration of single invocation 📝 - Benchmark Source	18.98 μs	79.88 μs	4.21	0.02	False
ctor_capacity - Duration of single invocation 📝 - Benchmark Source	5.52 μs	10.16 μs	1.84	0.05	False
Append_Char_Capacity - Duration of single invocation 📝 - Benchmark Source	284.55 μs	420.04 μs	1.48	0.48	False
ctor_string - Duration of single invocation 📝 - Benchmark Source	14.37 μs	72.14 μs	5.02	0.01	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Tests.Perf_StringBuilder*'

Payloads

Baseline
Compare

System.Text.Tests.Perf_StringBuilder.ToString_MultipleSegments(length: 100000)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 96.64492912371134 > 22.19193447327905.
IsChangePoint: Marked as a change because one of 7/17/2023 4:07:27 PM, 7/19/2023 6:59:29 PM falls between 7/10/2023 2:17:11 PM and 7/19/2023 6:59:29 PM.
IsRegressionStdDev: Marked as regression because -8.8614939204825 (T) = (0 -89009.10485155522) / Math.Sqrt((301034.7288344912 / (15)) + (116611624.63049658 / (2))) is less than -2.131449545559758 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (15) + (2) - 2, .025) and -3.172435378856418 = (21332.650303610182 - 89009.10485155522) / 21332.650303610182 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Tests.Perf_StringBuilder.ToString_SingleSegment(length: 100000)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 79.88275290697675 > 20.413894121914147.
IsChangePoint: Marked as a change because one of 7/17/2023 4:07:27 PM, 7/19/2023 6:59:29 PM falls between 7/10/2023 2:17:11 PM and 7/19/2023 6:59:29 PM.
IsRegressionStdDev: Marked as regression because -29.62250119071027 (T) = (0 -77905.38003955394) / Math.Sqrt((50497.475381388256 / (15)) + (7820006.9136397075 / (2))) is less than -2.131449545559758 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (15) + (2) - 2, .025) and -3.03541094841131 = (19305.439033470506 - 77905.38003955394) / 19305.439033470506 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Tests.Perf_StringBuilder.ctor_capacity(length: 100000)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 10.163017519393357 > 5.795809405674804.
IsChangePoint: Marked as a change because one of 4/25/2023 1:02:54 PM, 7/17/2023 4:07:27 PM, 7/19/2023 6:59:29 PM falls between 7/10/2023 2:17:11 PM and 7/19/2023 6:59:29 PM.
IsRegressionStdDev: Marked as regression because -135.76333695931683 (T) = (0 -10131.36906001631) / Math.Sqrt((2180.440353936697 / (15)) + (2003.2499618812392 / (2))) is less than -2.131449545559758 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (15) + (2) - 2, .025) and -0.8309336596506445 = (5533.444101928548 - 10131.36906001631) / 5533.444101928548 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Tests.Perf_StringBuilder.Append_Char_Capacity(length: 100000)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 420.04497722672073 > 300.31783359375.
IsChangePoint: Marked as a change because one of 3/25/2023 12:18:49 AM, 3/30/2023 5:15:39 PM, 5/18/2023 7:10:13 PM, 6/1/2023 5:52:46 AM, 6/28/2023 11:43:34 AM, 7/17/2023 4:07:27 PM, 7/19/2023 6:59:29 PM falls between 7/10/2023 2:17:11 PM and 7/19/2023 6:59:29 PM.
IsRegressionStdDev: Marked as regression because -18.9499134774535 (T) = (0 -420434.1379859472) / Math.Sqrt((781893432.2642138 / (15)) + (302892.19304345635 / (2))) is less than -2.131449545559758 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (15) + (2) - 2, .025) and -0.48343140528585415 = (283420.00613431155 - 420434.1379859472) / 283420.00613431155 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Tests.Perf_StringBuilder.ctor_string(length: 100000)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 72.13512505671508 > 15.111548469023417.
IsChangePoint: Marked as a change because one of 7/17/2023 4:07:27 PM, 7/19/2023 6:59:29 PM falls between 7/10/2023 2:17:11 PM and 7/19/2023 6:59:29 PM.
IsRegressionStdDev: Marked as regression because -71.20252621282843 (T) = (0 -72955.43077953125) / Math.Sqrt((41195.66829324731 / (15)) + (1345802.9577699201 / (2))) is less than -2.131449545559758 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (15) + (2) - 2, .025) and -4.056335134774942 = (14428.51963624411 - 72955.43077953125) / 14428.51963624411 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name	Value
Architecture	arm64
OS	Windows 10.0.19041
Queue	SurfaceWindows
Baseline	bb38848e9877dc0b43860dc219ebe36fc04a6bb3
Compare	16dc5330051a97aec4ba89d21263cdccc7b7552c
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Diagnostics.Perf_Process

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio
GetProcesses - Duration of single invocation 📝 - Benchmark Source	1.07 ms	2.02 ms	1.89	0.64	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Diagnostics.Perf_Process*'

Payloads

Baseline
Compare

System.Diagnostics.Perf_Process.GetProcesses

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.0161652529761906 > 1.1468048437500005.
IsChangePoint: Marked as a change because one of 4/8/2023 9:16:19 PM, 4/27/2023 1:05:59 AM, 7/17/2023 4:07:27 PM, 7/19/2023 6:59:29 PM falls between 7/10/2023 2:17:11 PM and 7/19/2023 6:59:29 PM.
IsRegressionStdDev: Marked as regression because -11.027556022158558 (T) = (0 -1960596.8091804031) / Math.Sqrt((30159161263.010315 / (15)) + (6175703891.771193 / (2))) is less than -2.131449545559758 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (15) + (2) - 2, .025) and -0.6711666399223366 = (1173190.4900109286 - 1960596.8091804031) / 1173190.4900109286 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name	Value
Architecture	arm64
OS	Windows 10.0.19041
Queue	SurfaceWindows
Baseline	bb38848e9877dc0b43860dc219ebe36fc04a6bb3
Compare	16dc5330051a97aec4ba89d21263cdccc7b7552c
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Tests.Perf_GC<Byte>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector
AllocateUninitializedArray - Duration of single invocation	384.51 ns	1.72 μs	4.47	0.09	False
AllocateArray - Duration of single invocation	775.27 ns	1.88 μs	2.42	0.10	False
AllocateUninitializedArray - Duration of single invocation	155.52 ns	472.17 ns	3.04	0.11	False
AllocateArray - Duration of single invocation	178.93 ns	470.77 ns	2.63	0.15	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_GC&lt;Byte&gt;*'

Payloads

Baseline
Compare

System.Tests.Perf_GC<Byte>.AllocateUninitializedArray(length: 10000, pinned: True)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.7177633974690083 > 399.6578134529415.
IsChangePoint: Marked as a change because one of 7/17/2023 4:07:27 PM, 7/19/2023 6:59:29 PM falls between 7/10/2023 2:17:11 PM and 7/19/2023 6:59:29 PM.
IsRegressionStdDev: Marked as regression because -256.67053966083284 (T) = (0 -1722.6061554861167) / Math.Sqrt((56.16090401014884 / (15)) + (46.90461042453306 / (2))) is less than -2.131449545559758 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (15) + (2) - 2, .025) and -3.485184576382786 = (384.0658341145383 - 1722.6061554861167) / 384.0658341145383 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_GC<Byte>.AllocateArray(length: 10000, pinned: True)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.877226064390347 > 814.3475494617968.
IsChangePoint: Marked as a change because one of 7/17/2023 4:07:27 PM, 7/19/2023 6:59:29 PM falls between 7/10/2023 2:17:11 PM and 7/19/2023 6:59:29 PM.
IsRegressionStdDev: Marked as regression because -440.67129718721765 (T) = (0 -1874.8971003547222) / Math.Sqrt((11.847738386678492 / (15)) + (10.848146958465723 / (2))) is less than -2.131449545559758 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (15) + (2) - 2, .025) and -1.414850080560727 = (776.4031048748882 - 1874.8971003547222) / 776.4031048748882 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_GC<Byte>.AllocateUninitializedArray(length: 1000, pinned: True)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 472.16513668101777 > 162.20111444090492.
IsChangePoint: Marked as a change because one of 5/23/2023 1:41:12 PM, 6/8/2023 5:31:08 AM, 7/17/2023 4:07:27 PM, 7/19/2023 6:59:29 PM falls between 7/10/2023 2:17:11 PM and 7/19/2023 6:59:29 PM.
IsRegressionStdDev: Marked as regression because -21.318217154468464 (T) = (0 -457.9533767902344) / Math.Sqrt((1.2117547655583252 / (15)) + (403.94823838655543 / (2))) is less than -2.131449545559758 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (15) + (2) - 2, .025) and -1.9559985600646497 = (154.92340996952944 - 457.9533767902344) / 154.92340996952944 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_GC<Byte>.AllocateArray(length: 1000, pinned: True)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 470.7704826208066 > 187.7223194932085.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/23/2023 1:41:12 PM, 6/8/2023 5:31:08 AM, 7/17/2023 4:07:27 PM, 7/19/2023 6:59:29 PM falls between 7/10/2023 2:17:11 PM and 7/19/2023 6:59:29 PM.
IsRegressionStdDev: Marked as regression because -115.38793741220125 (T) = (0 -468.94846461482723) / Math.Sqrt((43.88253710633865 / (15)) + (6.639499228225897 / (2))) is less than -2.131449545559758 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (15) + (2) - 2, .025) and -1.5967833600469057 = (180.5882122590144 - 468.94846461482723) / 180.5882122590144 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name	Value
Architecture	arm64
OS	Windows 10.0.19041
Queue	SurfaceWindows
Baseline	bb38848e9877dc0b43860dc219ebe36fc04a6bb3
Compare	16dc5330051a97aec4ba89d21263cdccc7b7552c
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Tests.Perf_GC<Char>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector
AllocateArray - Duration of single invocation	1.47 μs	2.28 μs	1.55	0.07	False
AllocateArray - Duration of single invocation	235.63 ns	797.53 ns	3.38	0.11	False
AllocateUninitializedArray - Duration of single invocation	186.48 ns	788.87 ns	4.23	0.09	False
AllocateUninitializedArray - Duration of single invocation	621.92 ns	2.38 μs	3.83	0.03	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_GC&lt;Char&gt;*'

Payloads

Baseline
Compare

System.Tests.Perf_GC<Char>.AllocateArray(length: 10000, pinned: True)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.2832381637321504 > 1.5367268810257242.
IsChangePoint: Marked as a change because one of 4/26/2023 8:38:54 PM, 7/17/2023 4:07:27 PM, 7/19/2023 6:59:29 PM falls between 7/10/2023 2:17:11 PM and 7/19/2023 6:59:29 PM.
IsRegressionStdDev: Marked as regression because -55.78707820712384 (T) = (0 -2298.0090499132884) / Math.Sqrt((63.27093197615269 / (15)) + (436.35815715226346 / (2))) is less than -2.131449545559758 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (15) + (2) - 2, .025) and -0.56747575912705 = (1466.0571536960056 - 2298.0090499132884) / 1466.0571536960056 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_GC<Char>.AllocateArray(length: 1000, pinned: True)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 797.532396788991 > 250.09650954960767.
IsChangePoint: Marked as a change because one of 7/17/2023 4:07:27 PM, 7/19/2023 6:59:29 PM falls between 7/10/2023 2:17:11 PM and 7/19/2023 6:59:29 PM.
IsRegressionStdDev: Marked as regression because -164.6218631515267 (T) = (0 -800.9410409499897) / Math.Sqrt((1.3949408754027728 / (15)) + (23.237710032619965 / (2))) is less than -2.131449545559758 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (15) + (2) - 2, .025) and -2.371495773855683 = (237.56252259335443 - 800.9410409499897) / 237.56252259335443 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_GC<Char>.AllocateUninitializedArray(length: 1000, pinned: True)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 788.8731905186607 > 194.27387716710592.
IsChangePoint: Marked as a change because one of 7/17/2023 4:07:27 PM, 7/19/2023 6:59:29 PM falls between 7/10/2023 2:17:11 PM and 7/19/2023 6:59:29 PM.
IsRegressionStdDev: Marked as regression because -53.6701390096785 (T) = (0 -777.8408674296243) / Math.Sqrt((1.1605997238051653 / (15)) + (243.4243054817749 / (2))) is less than -2.131449545559758 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (15) + (2) - 2, .025) and -3.192164061129945 = (185.54638036278737 - 777.8408674296243) / 185.54638036278737 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_GC<Char>.AllocateUninitializedArray(length: 10000, pinned: True)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.3790466059342608 > 660.7370015602506.
IsChangePoint: Marked as a change because one of 7/17/2023 4:07:27 PM, 7/19/2023 6:59:29 PM falls between 7/10/2023 2:17:11 PM and 7/19/2023 6:59:29 PM.
IsRegressionStdDev: Marked as regression because -29.929305140791712 (T) = (0 -2322.4811732723774) / Math.Sqrt((44.988564752216064 / (15)) + (6399.296344452154 / (2))) is less than -2.131449545559758 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (15) + (2) - 2, .025) and -2.6939609501352844 = (628.7238020714109 - 2322.4811732723774) / 628.7238020714109 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

The text was updated successfully, but these errors were encountered:

performanceautofiler · 2023-07-20T08:20:41Z

Run Information

Name	Value
Architecture	arm64
OS	Windows 10.0.19041
Queue	SurfaceWindows
Baseline	bb38848e9877dc0b43860dc219ebe36fc04a6bb3
Compare	16dc5330051a97aec4ba89d21263cdccc7b7552c
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.IO.Tests.Perf_File

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio
ReadAllBytesAsync - Duration of single invocation 📝 - Benchmark Source	274.91 μs	577.99 μs	2.10	0.27	False
ReadAllBytes - Duration of single invocation 📝 - Benchmark Source	202.83 μs	479.94 μs	2.37	0.29	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.IO.Tests.Perf_File*'

Payloads

Baseline
Compare

System.IO.Tests.Perf_File.ReadAllBytesAsync(size: 1048576)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 577.9861458333332 > 290.3141094388058.
IsChangePoint: Marked as a change because one of 4/25/2023 1:02:54 PM, 7/17/2023 4:07:27 PM, 7/19/2023 6:59:29 PM falls between 7/10/2023 2:17:11 PM and 7/19/2023 6:59:29 PM.
IsRegressionStdDev: Marked as regression because -15.284701840479862 (T) = (0 -560749.06445802) / Math.Sqrt((487603873.49851245 / (15)) + (594233948.6783401 / (2))) is less than -2.131449545559758 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (15) + (2) - 2, .025) and -0.9797180198262446 = (283246.93660526245 - 560749.06445802) / 283246.93660526245 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.IO.Tests.Perf_File.ReadAllBytes(size: 1048576)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 479.94292338709687 > 231.82580033365883.
IsChangePoint: Marked as a change because one of 7/17/2023 4:07:27 PM, 7/19/2023 6:59:29 PM falls between 7/10/2023 2:17:11 PM and 7/19/2023 6:59:29 PM.
IsRegressionStdDev: Marked as regression because -23.577822242410264 (T) = (0 -490412.2564728353) / Math.Sqrt((491765563.06122065 / (15)) + (219213870.52027768 / (2))) is less than -2.131449545559758 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (15) + (2) - 2, .025) and -1.3457605174110605 = (209063.22398762486 - 490412.2564728353) / 209063.22398762486 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name	Value
Architecture	arm64
OS	Windows 10.0.19041
Queue	SurfaceWindows
Baseline	bb38848e9877dc0b43860dc219ebe36fc04a6bb3
Compare	16dc5330051a97aec4ba89d21263cdccc7b7552c
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Numerics.Tests.Perf_Matrix4x4

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio
CreateFromYawPitchRollBenchmarkBenchmark - Duration of single invocation 📝 - Benchmark Source	72.09 ns	84.47 ns	1.17	0.03	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Numerics.Tests.Perf_Matrix4x4*'

Payloads

Baseline
Compare

System.Numerics.Tests.Perf_Matrix4x4.CreateFromYawPitchRollBenchmarkBenchmark

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 84.46788241404712 > 75.75313546496682.
IsChangePoint: Marked as a change because one of 7/17/2023 4:07:27 PM, 7/19/2023 6:59:29 PM falls between 7/10/2023 2:17:11 PM and 7/19/2023 6:59:29 PM.
IsRegressionStdDev: Marked as regression because -7.645247215071372 (T) = (0 -86.27122724084657) / Math.Sqrt((0.7910449094041205 / (15)) + (6.504105128688698 / (2))) is less than -2.131449545559758 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (15) + (2) - 2, .025) and -0.19203818135424594 = (72.37287243839447 - 86.27122724084657) / 72.37287243839447 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name	Value
Architecture	arm64
OS	Windows 10.0.19041
Queue	SurfaceWindows
Baseline	bb38848e9877dc0b43860dc219ebe36fc04a6bb3
Compare	16dc5330051a97aec4ba89d21263cdccc7b7552c
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in MicroBenchmarks.Serializers.Json_ToString<CollectionsOfPrimitives>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio
SystemTextJson_SourceGen_ - Duration of single invocation	139.24 μs	175.28 μs	1.26	0.03	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'MicroBenchmarks.Serializers.Json_ToString&lt;CollectionsOfPrimitives&gt;*'

Payloads

Baseline
Compare

MicroBenchmarks.Serializers.Json_ToString<CollectionsOfPrimitives>.SystemTextJson_SourceGen_

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 175.27994382022473 > 145.7066131121954.
IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 7/17/2023 4:07:27 PM, 7/19/2023 6:59:29 PM falls between 7/10/2023 2:17:11 PM and 7/19/2023 6:59:29 PM.
IsRegressionStdDev: Marked as regression because -9.778227603968837 (T) = (0 -171882.92275646652) / Math.Sqrt((185977.0733591263 / (15)) + (23079504.215233866 / (2))) is less than -2.131449545559758 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (15) + (2) - 2, .025) and -0.2397050631655811 = (138648.23808782734 - 171882.92275646652) / 138648.23808782734 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

cincuranet · 2023-07-20T16:32:40Z

Best guess for the commit range c4ff86d...4f933e1 is #87374.

cc @MichalPetryka

AndyAyersMS · 2023-07-20T16:39:56Z

This is also affecting x64:

It's possible that the intrinsifcation here broke static PGO ingestion, but that's just a guess.

AndyAyersMS · 2023-07-20T23:04:34Z

It's possible that the intrinsifcation here broke static PGO ingestion, but that's just a guess.

No, that doesn't seem to be the case. Main method in the benchmark above is TranscodeToUtf8 and

base

82.00%   4.265E+07   Tier-1   [System.Private.CoreLib]Utf8Utility.TranscodeToUtf8(wchar*,int32,unsigned int8*,int32,wchar*&,unsigned int8*&)
08.54%   4.44E+06    Tier-1   [System.Private.CoreLib]Utf16Utility.GetPointerToFirstInvalidChar(wchar*,int32,int64&,int32&)
07.25%   3.77E+06    native   coreclr.dll

This method does not end up depending on GetArrayDataReference directly.

Another possibility here is the BDN version update we took with dotnet/performance#3120.

AndyAyersMS · 2023-07-20T23:19:24Z

Method	BDN	Toolchain	Input	Mean	Error	StdDev	Median	Min	Max	Ratio	Gen0	Gen1	Gen2	Allocated	Alloc Ratio
GetBytes	Old BDN	\base-rel\corerun.exe	Cyrillic	123.3 us	1.10 us	0.97 us	123.3 us	121.9 us	125.0 us	1.00	31.9882	31.9882	31.9882	99.89 KB	1.00
GetBytes	Old BDN	\diff-rel\corerun.exe	Cyrillic	109.4 us	1.35 us	1.26 us	109.4 us	106.0 us	111.3 us	0.89	31.8966	31.8966	31.8966	99.89 KB	1.00
GetBytes	New BDN	\base-rel\corerun.exe	Cyrillic	155.06 us	2.152 us	2.013 us	155.07 us	152.11 us	158.99 us	1.00	0.00	32.1602	32.1602	32.1602	99.89 KB
GetBytes	New BDN	\diff-rel\corerun.exe	Cyrillic	154.99 us	1.623 us	1.519 us	155.06 us	152.45 us	157.82 us	1.00	0.02	32.1782	32.1782	32.1782	99.89 KB

Here

Old BDN is BenchmarkDotNet v0.13.2.2052-nightly
New BDN is BenchmarkDotNet v0.13.7-nightly.20230717.35

So that seems to be the most likely explanation for the "regression"

cc @cincuranet

AndyAyersMS · 2023-07-20T23:32:36Z

Also note the operation count dropped quite a bit. Note this can be an effect (not a cause) if the benchmark is indeed running more slowly, but it can also impact the timing.

ghost · 2023-07-20T23:39:43Z

Tagging subscribers to this area: @dotnet/runtime-infrastructure
See info in area-owners.md if you want to be subscribed.

Issue Details

Run Information

Name	Value
Architecture	arm64
OS	Windows 10.0.19041
Queue	SurfaceWindows
Baseline	bb38848e9877dc0b43860dc219ebe36fc04a6bb3
Compare	16dc5330051a97aec4ba89d21263cdccc7b7552c
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Text.Perf_Utf8Encoding

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector
GetBytes - Duration of single invocation 📝 - Benchmark Source	134.64 μs	163.41 μs	1.21	0.03	False
GetBytes - Duration of single invocation 📝 - Benchmark Source	144.34 μs	190.21 μs	1.32	0.01	False
GetString - Duration of single invocation 📝 - Benchmark Source	319.91 μs	361.24 μs	1.13	0.00	False
GetString - Duration of single invocation 📝 - Benchmark Source	200.01 μs	236.03 μs	1.18	0.00	False
GetBytes - Duration of single invocation 📝 - Benchmark Source	159.97 μs	208.18 μs	1.30	0.02	False
GetString - Duration of single invocation 📝 - Benchmark Source	187.68 μs	251.38 μs	1.34	0.05	False
GetBytes - Duration of single invocation 📝 - Benchmark Source	203.39 μs	238.89 μs	1.17	0.02	False
GetString - Duration of single invocation 📝 - Benchmark Source	44.30 μs	135.22 μs	3.05	0.01	False
GetBytes - Duration of single invocation 📝 - Benchmark Source	42.38 μs	78.88 μs	1.86	0.04	False
GetString - Duration of single invocation 📝 - Benchmark Source	261.72 μs	306.76 μs	1.17	0.01	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Perf_Utf8Encoding*'

Payloads

Baseline
Compare

System.Text.Perf_Utf8Encoding.GetBytes(Input: Cyrillic)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 163.41458333333333 > 141.77467636148876.
IsChangePoint: Marked as a change because one of 3/6/2023 6:03:21 AM, 4/4/2023 10:11:10 PM, 5/19/2023 4:22:12 AM, 7/17/2023 4:07:27 PM, 7/19/2023 6:59:29 PM falls between 7/10/2023 2:17:11 PM and 7/19/2023 6:59:29 PM.
IsRegressionStdDev: Marked as regression because -156.9943601397984 (T) = (0 -163313.67105263157) / Math.Sqrt((332399.08210140927 / (15)) + (20366.576792853983 / (2))) is less than -2.131449545559758 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (15) + (2) - 2, .025) and -0.20901931770273424 = (135079.45544074927 - 163313.67105263157) / 135079.45544074927 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Perf_Utf8Encoding.GetBytes(Input: EnglishMostlyAscii)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 190.21237931910568 > 151.46738830222137.
IsChangePoint: Marked as a change because one of 5/6/2023 11:12:53 AM, 7/17/2023 4:07:27 PM, 7/19/2023 6:59:29 PM falls between 7/10/2023 2:17:11 PM and 7/19/2023 6:59:29 PM.
IsRegressionStdDev: Marked as regression because -57.93111328415174 (T) = (0 -191002.3032508227) / Math.Sqrt((264013.2822441723 / (15)) + (1247959.6357985316 / (2))) is less than -2.131449545559758 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (15) + (2) - 2, .025) and -0.32089916063131374 = (144600.21547711076 - 191002.3032508227) / 144600.21547711076 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Perf_Utf8Encoding.GetString(Input: Greek)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 361.24470930232553 > 335.2484196428572.
IsChangePoint: Marked as a change because one of 3/16/2023 8:14:39 PM, 4/18/2023 4:36:42 PM, 6/22/2023 12:06:43 AM, 7/17/2023 4:07:27 PM, 7/19/2023 6:59:29 PM falls between 7/10/2023 2:17:11 PM and 7/19/2023 6:59:29 PM.
IsRegressionStdDev: Marked as regression because -14.118172520751896 (T) = (0 -364439.96422182466) / Math.Sqrt((156432.7000792867 / (15)) + (20419308.001166407 / (2))) is less than -2.131449545559758 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (15) + (2) - 2, .025) and -0.14135101329335703 = (319305.770071765 - 364439.96422182466) / 319305.770071765 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Perf_Utf8Encoding.GetString(Input: Cyrillic)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 236.02786796536802 > 209.01690668706294.
IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 6/22/2023 12:06:43 AM, 7/17/2023 4:07:27 PM, 7/19/2023 6:59:29 PM falls between 7/10/2023 2:17:11 PM and 7/19/2023 6:59:29 PM.
IsRegressionStdDev: Marked as regression because -31.45956270973919 (T) = (0 -237201.51176948054) / Math.Sqrt((948388.27929041 / (15)) + (2754879.557863424 / (2))) is less than -2.131449545559758 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (15) + (2) - 2, .025) and -0.18932988111238652 = (199441.31189878515 - 237201.51176948054) / 199441.31189878515 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Perf_Utf8Encoding.GetBytes(Input: Chinese)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 208.17916666666665 > 167.74164674908423.
IsChangePoint: Marked as a change because one of 5/6/2023 11:12:53 AM, 5/19/2023 4:22:12 AM, 6/22/2023 12:06:43 AM, 7/17/2023 4:07:27 PM, 7/19/2023 6:59:29 PM falls between 7/10/2023 2:17:11 PM and 7/19/2023 6:59:29 PM.
IsRegressionStdDev: Marked as regression because -105.10239953388513 (T) = (0 -207929.19060106558) / Math.Sqrt((2181557.5854358426 / (15)) + (124976.06674677305 / (2))) is less than -2.131449545559758 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (15) + (2) - 2, .025) and -0.2995270615430891 = (160003.7404024242 - 207929.19060106558) / 160003.7404024242 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Perf_Utf8Encoding.GetString(Input: EnglishMostlyAscii)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 251.37670546558707 > 199.26802149662032.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 7/17/2023 4:07:27 PM, 7/19/2023 6:59:29 PM falls between 7/10/2023 2:17:11 PM and 7/19/2023 6:59:29 PM.
IsRegressionStdDev: Marked as regression because -49.931345545245065 (T) = (0 -252501.08962795482) / Math.Sqrt((3578319.5423276178 / (15)) + (2528479.4891668903 / (2))) is less than -2.131449545559758 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (15) + (2) - 2, .025) and -0.31998419269850836 = (191290.99501695886 - 252501.08962795482) / 191290.99501695886 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Perf_Utf8Encoding.GetBytes(Input: Greek)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 238.88602810650892 > 213.3333234577922.
IsChangePoint: Marked as a change because one of 3/6/2023 6:03:21 AM, 3/14/2023 8:42:09 PM, 3/21/2023 2:53:21 PM, 4/4/2023 10:11:10 PM, 5/19/2023 4:22:12 AM, 7/17/2023 4:07:27 PM, 7/19/2023 6:59:29 PM falls between 7/10/2023 2:17:11 PM and 7/19/2023 6:59:29 PM.
IsRegressionStdDev: Marked as regression because -46.44101491427244 (T) = (0 -238147.19893087685) / Math.Sqrt((337750.0174463117 / (15)) + (1091737.1015303673 / (2))) is less than -2.131449545559758 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (15) + (2) - 2, .025) and -0.17236098584785833 = (203134.70151742332 - 238147.19893087685) / 203134.70151742332 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Perf_Utf8Encoding.GetString(Input: EnglishAllAscii)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 135.2222883064516 > 46.67656657750623.
IsChangePoint: Marked as a change because one of 7/17/2023 4:07:27 PM, 7/19/2023 6:59:29 PM falls between 7/10/2023 2:17:11 PM and 7/19/2023 6:59:29 PM.
IsRegressionStdDev: Marked as regression because -10.519272418769509 (T) = (0 -144750.35992703532) / Math.Sqrt((105388.31582920565 / (15)) + (181568297.61394644 / (2))) is less than -2.131449545559758 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (15) + (2) - 2, .025) and -2.2514945147506236 = (44518.10060584927 - 144750.35992703532) / 44518.10060584927 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Perf_Utf8Encoding.GetBytes(Input: EnglishAllAscii)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 78.88215829439251 > 44.312702125.
IsChangePoint: Marked as a change because one of 4/11/2023 5:52:15 AM, 7/17/2023 4:07:27 PM, 7/19/2023 6:59:29 PM falls between 7/10/2023 2:17:11 PM and 7/19/2023 6:59:29 PM.
IsRegressionStdDev: Marked as regression because -20.02921160723858 (T) = (0 -77139.65478125423) / Math.Sqrt((224515.86856747864 / (15)) + (6072636.986598566 / (2))) is less than -2.131449545559758 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (15) + (2) - 2, .025) and -0.8300022207196299 = (42152.765667639374 - 77139.65478125423) / 42152.765667639374 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Perf_Utf8Encoding.GetString(Input: Chinese)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 306.76269403594773 > 275.3442643065268.
IsChangePoint: Marked as a change because one of 3/16/2023 8:14:39 PM, 4/25/2023 6:46:56 PM, 7/17/2023 4:07:27 PM, 7/19/2023 6:59:29 PM falls between 7/10/2023 2:17:11 PM and 7/19/2023 6:59:29 PM.
IsRegressionStdDev: Marked as regression because -108.94988157600964 (T) = (0 -306379.07110760966) / Math.Sqrt((241114.1247489303 / (15)) + (294333.10229330295 / (2))) is less than -2.131449545559758 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (15) + (2) - 2, .025) and -0.16778130939409266 = (262359.9715485903 - 306379.07110760966) / 262359.9715485903 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name	Value
Architecture	arm64
OS	Windows 10.0.19041
Queue	SurfaceWindows
Baseline	bb38848e9877dc0b43860dc219ebe36fc04a6bb3
Compare	16dc5330051a97aec4ba89d21263cdccc7b7552c
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Text.Tests.Perf_StringBuilder

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector
ToString_MultipleSegments - Duration of single invocation 📝 - Benchmark Source	21.13 μs	96.64 μs	4.57	0.07	False
ToString_SingleSegment - Duration of single invocation 📝 - Benchmark Source	18.98 μs	79.88 μs	4.21	0.02	False
ctor_capacity - Duration of single invocation 📝 - Benchmark Source	5.52 μs	10.16 μs	1.84	0.05	False
Append_Char_Capacity - Duration of single invocation 📝 - Benchmark Source	284.55 μs	420.04 μs	1.48	0.48	False
ctor_string - Duration of single invocation 📝 - Benchmark Source	14.37 μs	72.14 μs	5.02	0.01	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Tests.Perf_StringBuilder*'

Payloads

Baseline
Compare

System.Text.Tests.Perf_StringBuilder.ToString_MultipleSegments(length: 100000)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 96.64492912371134 > 22.19193447327905.
IsChangePoint: Marked as a change because one of 7/17/2023 4:07:27 PM, 7/19/2023 6:59:29 PM falls between 7/10/2023 2:17:11 PM and 7/19/2023 6:59:29 PM.
IsRegressionStdDev: Marked as regression because -8.8614939204825 (T) = (0 -89009.10485155522) / Math.Sqrt((301034.7288344912 / (15)) + (116611624.63049658 / (2))) is less than -2.131449545559758 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (15) + (2) - 2, .025) and -3.172435378856418 = (21332.650303610182 - 89009.10485155522) / 21332.650303610182 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Tests.Perf_StringBuilder.ToString_SingleSegment(length: 100000)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 79.88275290697675 > 20.413894121914147.
IsChangePoint: Marked as a change because one of 7/17/2023 4:07:27 PM, 7/19/2023 6:59:29 PM falls between 7/10/2023 2:17:11 PM and 7/19/2023 6:59:29 PM.
IsRegressionStdDev: Marked as regression because -29.62250119071027 (T) = (0 -77905.38003955394) / Math.Sqrt((50497.475381388256 / (15)) + (7820006.9136397075 / (2))) is less than -2.131449545559758 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (15) + (2) - 2, .025) and -3.03541094841131 = (19305.439033470506 - 77905.38003955394) / 19305.439033470506 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Tests.Perf_StringBuilder.ctor_capacity(length: 100000)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 10.163017519393357 > 5.795809405674804.
IsChangePoint: Marked as a change because one of 4/25/2023 1:02:54 PM, 7/17/2023 4:07:27 PM, 7/19/2023 6:59:29 PM falls between 7/10/2023 2:17:11 PM and 7/19/2023 6:59:29 PM.
IsRegressionStdDev: Marked as regression because -135.76333695931683 (T) = (0 -10131.36906001631) / Math.Sqrt((2180.440353936697 / (15)) + (2003.2499618812392 / (2))) is less than -2.131449545559758 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (15) + (2) - 2, .025) and -0.8309336596506445 = (5533.444101928548 - 10131.36906001631) / 5533.444101928548 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Tests.Perf_StringBuilder.Append_Char_Capacity(length: 100000)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 420.04497722672073 > 300.31783359375.
IsChangePoint: Marked as a change because one of 3/25/2023 12:18:49 AM, 3/30/2023 5:15:39 PM, 5/18/2023 7:10:13 PM, 6/1/2023 5:52:46 AM, 6/28/2023 11:43:34 AM, 7/17/2023 4:07:27 PM, 7/19/2023 6:59:29 PM falls between 7/10/2023 2:17:11 PM and 7/19/2023 6:59:29 PM.
IsRegressionStdDev: Marked as regression because -18.9499134774535 (T) = (0 -420434.1379859472) / Math.Sqrt((781893432.2642138 / (15)) + (302892.19304345635 / (2))) is less than -2.131449545559758 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (15) + (2) - 2, .025) and -0.48343140528585415 = (283420.00613431155 - 420434.1379859472) / 283420.00613431155 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Tests.Perf_StringBuilder.ctor_string(length: 100000)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 72.13512505671508 > 15.111548469023417.
IsChangePoint: Marked as a change because one of 7/17/2023 4:07:27 PM, 7/19/2023 6:59:29 PM falls between 7/10/2023 2:17:11 PM and 7/19/2023 6:59:29 PM.
IsRegressionStdDev: Marked as regression because -71.20252621282843 (T) = (0 -72955.43077953125) / Math.Sqrt((41195.66829324731 / (15)) + (1345802.9577699201 / (2))) is less than -2.131449545559758 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (15) + (2) - 2, .025) and -4.056335134774942 = (14428.51963624411 - 72955.43077953125) / 14428.51963624411 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name	Value
Architecture	arm64
OS	Windows 10.0.19041
Queue	SurfaceWindows
Baseline	bb38848e9877dc0b43860dc219ebe36fc04a6bb3
Compare	16dc5330051a97aec4ba89d21263cdccc7b7552c
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Diagnostics.Perf_Process

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio
GetProcesses - Duration of single invocation 📝 - Benchmark Source	1.07 ms	2.02 ms	1.89	0.64	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Diagnostics.Perf_Process*'

Payloads

Baseline
Compare

System.Diagnostics.Perf_Process.GetProcesses

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.0161652529761906 > 1.1468048437500005.
IsChangePoint: Marked as a change because one of 4/8/2023 9:16:19 PM, 4/27/2023 1:05:59 AM, 7/17/2023 4:07:27 PM, 7/19/2023 6:59:29 PM falls between 7/10/2023 2:17:11 PM and 7/19/2023 6:59:29 PM.
IsRegressionStdDev: Marked as regression because -11.027556022158558 (T) = (0 -1960596.8091804031) / Math.Sqrt((30159161263.010315 / (15)) + (6175703891.771193 / (2))) is less than -2.131449545559758 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (15) + (2) - 2, .025) and -0.6711666399223366 = (1173190.4900109286 - 1960596.8091804031) / 1173190.4900109286 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name	Value
Architecture	arm64
OS	Windows 10.0.19041
Queue	SurfaceWindows
Baseline	bb38848e9877dc0b43860dc219ebe36fc04a6bb3
Compare	16dc5330051a97aec4ba89d21263cdccc7b7552c
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Tests.Perf_GC<Byte>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector
AllocateUninitializedArray - Duration of single invocation	384.51 ns	1.72 μs	4.47	0.09	False
AllocateArray - Duration of single invocation	775.27 ns	1.88 μs	2.42	0.10	False
AllocateUninitializedArray - Duration of single invocation	155.52 ns	472.17 ns	3.04	0.11	False
AllocateArray - Duration of single invocation	178.93 ns	470.77 ns	2.63	0.15	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_GC&lt;Byte&gt;*'

Payloads

Baseline
Compare

System.Tests.Perf_GC<Byte>.AllocateUninitializedArray(length: 10000, pinned: True)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.7177633974690083 > 399.6578134529415.
IsChangePoint: Marked as a change because one of 7/17/2023 4:07:27 PM, 7/19/2023 6:59:29 PM falls between 7/10/2023 2:17:11 PM and 7/19/2023 6:59:29 PM.
IsRegressionStdDev: Marked as regression because -256.67053966083284 (T) = (0 -1722.6061554861167) / Math.Sqrt((56.16090401014884 / (15)) + (46.90461042453306 / (2))) is less than -2.131449545559758 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (15) + (2) - 2, .025) and -3.485184576382786 = (384.0658341145383 - 1722.6061554861167) / 384.0658341145383 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_GC<Byte>.AllocateArray(length: 10000, pinned: True)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.877226064390347 > 814.3475494617968.
IsChangePoint: Marked as a change because one of 7/17/2023 4:07:27 PM, 7/19/2023 6:59:29 PM falls between 7/10/2023 2:17:11 PM and 7/19/2023 6:59:29 PM.
IsRegressionStdDev: Marked as regression because -440.67129718721765 (T) = (0 -1874.8971003547222) / Math.Sqrt((11.847738386678492 / (15)) + (10.848146958465723 / (2))) is less than -2.131449545559758 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (15) + (2) - 2, .025) and -1.414850080560727 = (776.4031048748882 - 1874.8971003547222) / 776.4031048748882 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_GC<Byte>.AllocateUninitializedArray(length: 1000, pinned: True)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 472.16513668101777 > 162.20111444090492.
IsChangePoint: Marked as a change because one of 5/23/2023 1:41:12 PM, 6/8/2023 5:31:08 AM, 7/17/2023 4:07:27 PM, 7/19/2023 6:59:29 PM falls between 7/10/2023 2:17:11 PM and 7/19/2023 6:59:29 PM.
IsRegressionStdDev: Marked as regression because -21.318217154468464 (T) = (0 -457.9533767902344) / Math.Sqrt((1.2117547655583252 / (15)) + (403.94823838655543 / (2))) is less than -2.131449545559758 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (15) + (2) - 2, .025) and -1.9559985600646497 = (154.92340996952944 - 457.9533767902344) / 154.92340996952944 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_GC<Byte>.AllocateArray(length: 1000, pinned: True)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 470.7704826208066 > 187.7223194932085.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/23/2023 1:41:12 PM, 6/8/2023 5:31:08 AM, 7/17/2023 4:07:27 PM, 7/19/2023 6:59:29 PM falls between 7/10/2023 2:17:11 PM and 7/19/2023 6:59:29 PM.
IsRegressionStdDev: Marked as regression because -115.38793741220125 (T) = (0 -468.94846461482723) / Math.Sqrt((43.88253710633865 / (15)) + (6.639499228225897 / (2))) is less than -2.131449545559758 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (15) + (2) - 2, .025) and -1.5967833600469057 = (180.5882122590144 - 468.94846461482723) / 180.5882122590144 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name	Value
Architecture	arm64
OS	Windows 10.0.19041
Queue	SurfaceWindows
Baseline	bb38848e9877dc0b43860dc219ebe36fc04a6bb3
Compare	16dc5330051a97aec4ba89d21263cdccc7b7552c
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Tests.Perf_GC<Char>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector
AllocateArray - Duration of single invocation	1.47 μs	2.28 μs	1.55	0.07	False
AllocateArray - Duration of single invocation	235.63 ns	797.53 ns	3.38	0.11	False
AllocateUninitializedArray - Duration of single invocation	186.48 ns	788.87 ns	4.23	0.09	False
AllocateUninitializedArray - Duration of single invocation	621.92 ns	2.38 μs	3.83	0.03	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_GC&lt;Char&gt;*'

Payloads

Baseline
Compare

System.Tests.Perf_GC<Char>.AllocateArray(length: 10000, pinned: True)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.2832381637321504 > 1.5367268810257242.
IsChangePoint: Marked as a change because one of 4/26/2023 8:38:54 PM, 7/17/2023 4:07:27 PM, 7/19/2023 6:59:29 PM falls between 7/10/2023 2:17:11 PM and 7/19/2023 6:59:29 PM.
IsRegressionStdDev: Marked as regression because -55.78707820712384 (T) = (0 -2298.0090499132884) / Math.Sqrt((63.27093197615269 / (15)) + (436.35815715226346 / (2))) is less than -2.131449545559758 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (15) + (2) - 2, .025) and -0.56747575912705 = (1466.0571536960056 - 2298.0090499132884) / 1466.0571536960056 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_GC<Char>.AllocateArray(length: 1000, pinned: True)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 797.532396788991 > 250.09650954960767.
IsChangePoint: Marked as a change because one of 7/17/2023 4:07:27 PM, 7/19/2023 6:59:29 PM falls between 7/10/2023 2:17:11 PM and 7/19/2023 6:59:29 PM.
IsRegressionStdDev: Marked as regression because -164.6218631515267 (T) = (0 -800.9410409499897) / Math.Sqrt((1.3949408754027728 / (15)) + (23.237710032619965 / (2))) is less than -2.131449545559758 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (15) + (2) - 2, .025) and -2.371495773855683 = (237.56252259335443 - 800.9410409499897) / 237.56252259335443 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_GC<Char>.AllocateUninitializedArray(length: 1000, pinned: True)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 788.8731905186607 > 194.27387716710592.
IsChangePoint: Marked as a change because one of 7/17/2023 4:07:27 PM, 7/19/2023 6:59:29 PM falls between 7/10/2023 2:17:11 PM and 7/19/2023 6:59:29 PM.
IsRegressionStdDev: Marked as regression because -53.6701390096785 (T) = (0 -777.8408674296243) / Math.Sqrt((1.1605997238051653 / (15)) + (243.4243054817749 / (2))) is less than -2.131449545559758 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (15) + (2) - 2, .025) and -3.192164061129945 = (185.54638036278737 - 777.8408674296243) / 185.54638036278737 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_GC<Char>.AllocateUninitializedArray(length: 10000, pinned: True)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.3790466059342608 > 660.7370015602506.
IsChangePoint: Marked as a change because one of 7/17/2023 4:07:27 PM, 7/19/2023 6:59:29 PM falls between 7/10/2023 2:17:11 PM and 7/19/2023 6:59:29 PM.
IsRegressionStdDev: Marked as regression because -29.929305140791712 (T) = (0 -2322.4811732723774) / Math.Sqrt((44.988564752216064 / (15)) + (6399.296344452154 / (2))) is less than -2.131449545559758 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (15) + (2) - 2, .025) and -2.6939609501352844 = (628.7238020714109 - 2322.4811732723774) / 628.7238020714109 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Author:	performanceautofiler[bot]
Assignees:	-
Labels:	`arch-arm64`, `os-windows`, `tenet-performance-benchmarks`, `area-Infrastructure`, `runtime-coreclr`, `PGO`, `needs-area-label`
Milestone:	-

performanceautofiler bot added arch-arm64 os-windows PGO runtime-coreclr specific to the CoreCLR runtime untriaged New issue has not been triaged by the area owner labels Jul 20, 2023

cincuranet removed the untriaged New issue has not been triaged by the area owner label Jul 20, 2023

cincuranet transferred this issue from dotnet/perf-autofiling-issues Jul 20, 2023

dotnet-issue-labeler bot added the needs-area-label An area label is needed to ensure this gets routed to the appropriate area owners label Jul 20, 2023

ghost added the untriaged New issue has not been triaged by the area owner label Jul 20, 2023

AndyAyersMS added tenet-performance-benchmarks Issue from performance benchmark area-Infrastructure and removed untriaged New issue has not been triaged by the area owner needs-area-label An area label is needed to ensure this gets routed to the appropriate area owners labels Jul 20, 2023

AndyAyersMS mentioned this issue Jul 23, 2023

Dynamic PGO Microbenchmark Regressions #87194

Closed

DrewScoggins mentioned this issue Sep 5, 2023

[Perf] 7.0 -> 8.0-RC1 Microbenchmark Comparison #91626

Closed

agocke closed this as completed Jun 18, 2024

github-actions bot locked and limited conversation to collaborators Jul 18, 2024

[Perf] Windows/arm64: 28 Regressions on 7/18/2023 1:13:00 AM #89259

[Perf] Windows/arm64: 28 Regressions on 7/18/2023 1:13:00 AM #89259

Comments

performanceautofiler bot commented Jul 20, 2023

Run Information

Regressions in System.Text.Perf_Utf8Encoding

Repro

Payloads

System.Text.Perf_Utf8Encoding.GetBytes(Input: Cyrillic)

ETL Files

Histogram

Description of detection logic

JIT Disasms

System.Text.Perf_Utf8Encoding.GetBytes(Input: EnglishMostlyAscii)

ETL Files

Histogram

Description of detection logic

JIT Disasms

System.Text.Perf_Utf8Encoding.GetString(Input: Greek)

ETL Files

Histogram

Description of detection logic

JIT Disasms

System.Text.Perf_Utf8Encoding.GetString(Input: Cyrillic)

ETL Files

Histogram

Description of detection logic

JIT Disasms

System.Text.Perf_Utf8Encoding.GetBytes(Input: Chinese)

ETL Files

Histogram

Description of detection logic

JIT Disasms

System.Text.Perf_Utf8Encoding.GetString(Input: EnglishMostlyAscii)

ETL Files

Histogram

Description of detection logic

JIT Disasms

System.Text.Perf_Utf8Encoding.GetBytes(Input: Greek)

ETL Files

Histogram

Description of detection logic

JIT Disasms

System.Text.Perf_Utf8Encoding.GetString(Input: EnglishAllAscii)

ETL Files

Histogram

Description of detection logic

JIT Disasms

System.Text.Perf_Utf8Encoding.GetBytes(Input: EnglishAllAscii)

ETL Files

Histogram

Description of detection logic

JIT Disasms

System.Text.Perf_Utf8Encoding.GetString(Input: Chinese)

ETL Files

Histogram

Description of detection logic

JIT Disasms

Docs

Run Information

Regressions in System.Text.Tests.Perf_StringBuilder

Repro

Payloads

System.Text.Tests.Perf_StringBuilder.ToString_MultipleSegments(length: 100000)

ETL Files

Histogram

Description of detection logic

JIT Disasms

System.Text.Tests.Perf_StringBuilder.ToString_SingleSegment(length: 100000)

ETL Files

Histogram

Description of detection logic

JIT Disasms

System.Text.Tests.Perf_StringBuilder.ctor_capacity(length: 100000)

ETL Files

Histogram

Description of detection logic

JIT Disasms

System.Text.Tests.Perf_StringBuilder.Append_Char_Capacity(length: 100000)

ETL Files