diff --git a/docs/_changelog/header/v0.13.2.md b/docs/_changelog/header/v0.13.2.md new file mode 100644 index 0000000000..12f650044a --- /dev/null +++ b/docs/_changelog/header/v0.13.2.md @@ -0,0 +1,254 @@ +## Highlights + +In BenchmarkDotNet v0.13.2, we have implemented support for: + +* .NET 7 +* NativeAOT +* .NET Framework 4.8.1 + +We have also introduced new features and improvements including: + +* Possibility to hide selected columns, +* Allocation Ratio column, +* Logging progress and estimated finish time, +* ARM64 support for `BenchmarkDotNet.Diagnostics.Windows` package, +* Printing Hardware Intrinsics information, +* Glob filters support for DisassemblyDiagnoser. + +Of course, this release includes dozens of other improvements and bug fixes! + +Our special thanks go to [@mawosoft](https://github.com/mawosoft), [@YegorStepanov](https://github.com/YegorStepanov) and [@radical](https://github.com/radical) who fixed a LOT of really nasty bugs. + +## Supported technologies + +### .NET 7 and .NET Framework 4.8.1 + +.NET 4.8.1 has been [released](https://devblogs.microsoft.com/dotnet/announcing-dotnet-framework-481/) earlier this month, while .NET 7 should land in autumn this year. Now you can use BenchmarkDotNet to compare both! + +```ini +BenchmarkDotNet=v0.13.1.1845-nightly, OS=Windows 11 (10.0.22622.575) +Microsoft SQ1 3.0 GHz, 1 CPU, 8 logical and 8 physical cores +.NET SDK=7.0.100-preview.6.22352.1 + [Host] : .NET 7.0.0 (7.0.22.32404), Arm64 RyuJIT AdvSIMD + Job-QJVIDT : .NET 7.0.0 (7.0.22.32404), Arm64 RyuJIT AdvSIMD + Job-FNNXOY : .NET Framework 4.8.1 (4.8.9032.0), Arm64 RyuJIT +``` + +| Method | Runtime | Mean | Allocated | +|-------------- |--------------------- |---------:|----------:| +| BinaryTrees_2 | .NET 7.0 | 193.6 ms | 227.33 MB | +| BinaryTrees_2 | .NET Framework 4.8.1 | 192.8 ms | 228.01 MB | + +Credit for adding .NET 7 support in [#1816](https://github.com/dotnet/BenchmarkDotNet/pull/1816) goes to [@am11](https://github.com/am11). [@adamsitnik](https://github.com/adamsitnik) implemented .NET 4.8.1 support in [#2044](https://github.com/dotnet/BenchmarkDotNet/pull/2044) and [#2067](https://github.com/dotnet/BenchmarkDotNet/pull/2067). Big thanks to [@MichalPetryka](https://github.com/MichalPetryka) who was using preview versions of BenchmarkDotNet and reported a bug related to .NET 4.8.1 support: [#2059](https://github.com/dotnet/BenchmarkDotNet/issues/2059) that got fixed before we released a new version. + +### NativeAOT + +We are really excited to see the experimental CoreRT project grow and become officially supported by Microsoft (under new name: NativeAOT)! You can read more about it [here](https://devblogs.microsoft.com/dotnet/announcing-dotnet-7-preview-3/#what-is-native-aot). Implementing and improving the support was a combined effort of multiple contributors that spawned across multiple repositories: +* [@MichalStrehovsky](https://github.com/MichalStrehovsky): [#66290 in dotnet/runtime](https://github.com/dotnet/runtime/pull/66290), [#2020 in dotnet/BenchmarkDotNet](https://github.com/dotnet/BenchmarkDotNet/pull/2020), [#2046 in dotnet/BenchmarkDotNet](https://github.com/dotnet/BenchmarkDotNet/pull/2046) +* [@hez2010](https://github.com/hez2010): [#66650 in dotnet/runtime](https://github.com/dotnet/runtime/pull/66650) +* [@Beau-Gosse-dev](https://github.com/Beau-Gosse-dev): [#1955 in dotnet/BenchmarkDotNet](https://github.com/dotnet/BenchmarkDotNet/pull/1955) +* [@adamsitnik](https://github.com/adamsitnik): [#1960 in dotnet/BenchmarkDotNet](https://github.com/dotnet/BenchmarkDotNet/pull/1960), [#1965 in dotnet/BenchmarkDotNet](https://github.com/dotnet/BenchmarkDotNet/pull/1965), [#1972 in dotnet/BenchmarkDotNet](https://github.com/dotnet/BenchmarkDotNet/pull/1972), [#1973 in dotnet/BenchmarkDotNet](https://github.com/dotnet/BenchmarkDotNet/pull/1973), [#1994 in dotnet/BenchmarkDotNet](https://github.com/dotnet/BenchmarkDotNet/pull/1994), [#1997 in dotnet/BenchmarkDotNet](https://github.com/dotnet/BenchmarkDotNet/pull/1997), [#2045 in dotnet/BenchmarkDotNet](https://github.com/dotnet/BenchmarkDotNet/pull/2045), [#2068 in dotnet/BenchmarkDotNet](https://github.com/dotnet/BenchmarkDotNet/pull/2068) +* [@kant2002](https://github.com/kant2002): [#1976 in dotnet/BenchmarkDotNet](https://github.com/dotnet/BenchmarkDotNet/pull/1976), [#1979 in dotnet/BenchmarkDotNet](https://github.com/dotnet/BenchmarkDotNet/pull/1979) +* [@jkotas](https://github.com/jkotas): [#68038 in dotnet/runtime](https://github.com/dotnet/runtime/pull/68038), [#68142 in dotnet/runtime](https://github.com/dotnet/runtime/pull/68142), [#68249 in dotnet/runtime](https://github.com/dotnet/runtime/pull/68249), [#68308 in dotnet/runtime](https://github.com/dotnet/runtime/pull/68308), [#68375 in dotnet/runtime](https://github.com/dotnet/runtime/pull/68375) +* [@MichalPetryka](https://github.com/MichalPetryka): [#2065 in dotnet/BenchmarkDotNet](https://github.com/dotnet/BenchmarkDotNet/pull/2065) + + +As every AOT solution, NativeAOT has some [limitations](https://github.com/dotnet/runtime/blob/main/src/coreclr/nativeaot/docs/limitations.md) like limited reflection support or lack of dynamic assembly loading. Because of that, the host process (what you run from command line) is never an AOT process, but just a regular .NET process. This process (called Host process) uses reflection to read benchmarks metadata (find all `[Benchmark]` methods etc.), generates a new project that references the benchmarks and compiles it using ILCompiler. The boilerplate code is not using reflection, so the project is built with `TrimmerDefaultAction=link` (we have greatly reduced build time thanks to that). Such compilation produces a native executable, which is later started by the Host process. This process (called Benchmark or Child process) performs the actual benchmarking and reports the results back to the Host process. By default BenchmarkDotNet uses the latest version of `Microsoft.DotNet.ILCompiler` to build the NativeAOT benchmark according to [this instructions](https://github.com/dotnet/runtime/blob/main/src/coreclr/nativeaot/docs/compiling.md). Moreover, BenchmarkDotNet by default uses current machines CPU features (change: [#1994](https://github.com/dotnet/BenchmarkDotNet/pull/1994), discussion: [#2061](https://github.com/dotnet/BenchmarkDotNet/issues/2061)) and if you don't like this behavior, you can [disable it](https://github.com/dotnet/BenchmarkDotNet/issues/2061#issuecomment-1203602177). + +This is why you need to: +- install [pre-requisites](https://github.com/dotnet/runtime/blob/main/src/coreclr/nativeaot/docs/prerequisites.md) required by NativeAOT compiler +- target .NET to be able to run NativeAOT benchmarks (example: `net7.0` in the .csproj file) +- run the app as a .NET process (example: `dotnet run -c Release -f net7.0`). +- specify the NativeAOT runtime in an explicit way, either by using console line arguments `--runtimes nativeaot7.0` (the recommended approach), or by using`[SimpleJob]` attribute or by using the fluent Job config API `Job.ShortRun.With(NativeAotRuntime.Net70)`: + +```cmd +dotnet run -c Release -f net7.0 --runtimes nativeaot7.0 +``` + +For more examples please go to [docs](https://benchmarkdotnet.org/articles/configs/toolchains.html#nativeaot). + +```ini +BenchmarkDotNet=v0.13.1.1845-nightly, OS=Windows 11 (10.0.22000.856/21H2) +AMD Ryzen Threadripper PRO 3945WX 12-Cores, 1 CPU, 24 logical and 12 physical cores +.NET SDK=7.0.100-rc.1.22423.16 + [Host] : .NET 7.0.0 (7.0.22.42223), X64 RyuJIT AVX2 + Job-KDVXET : .NET 7.0.0 (7.0.22.42223), X64 RyuJIT AVX2 + Job-HFRAGK : .NET 7.0.0-rc.1.22424.9, X64 NativeAOT AVX2 +``` + +| Method | Runtime | Mean | Ratio | Allocated | +|-------------- |-------------- |---------:|------:|----------:| +| BinaryTrees_2 | .NET 7.0 | 95.06 ms | 1.00 | 227.33 MB | +| BinaryTrees_2 | NativeAOT 7.0 | 90.32 ms | 0.96 | 227.33 MB | + + +Some of .NET features are not supported by Native AOT, that is why you may want to filter them out using new `[AotFilter]` attribute: + +```cs +[AotFilter("Currently not supported due to missing metadata.")] +public class Xml_FromStream +``` + +## New features and improvements + +### Hiding Columns + +In [#1621](https://github.com/dotnet/BenchmarkDotNet/pull/1621) [@marcnet80](https://github.com/marcnet80) has reduced the number of columns displayed when multiple runtimes are being compared. + +![](https://user-images.githubusercontent.com/6011991/99952390-defab180-2d7f-11eb-851c-ffef6fbb0647.png#mid) + +In [#1890](https://github.com/dotnet/BenchmarkDotNet/pull/1890) [@YegorStepanov](https://github.com/YegorStepanov) has implemented a set of new APIs that allow for hiding columns. It's also exposed via `-h` and `--hide` command line arguments. + +```cs +[MemoryDiagnoser] // adds Gen0, Gen1, Gen2 and Allocated Bytes columns +[HideColumns(Column.Gen0, Column.Gen1, Column.Gen2)] // dont display GenX columns +public class IntroHidingColumns +{ + [Benchmark] + public byte[] AllocateArray() => new byte[100_000]; +} +``` + +Sample results without `[HideColumns]`: + + +| Method | Mean | Error | StdDev | Gen0 | Gen1 | Gen2 | Allocated | +|-------------- |---------:|----------:|----------:|--------:|--------:|--------:|----------:| +| AllocateArray | 3.303 us | 0.0465 us | 0.0435 us | 31.2462 | 31.2462 | 31.2462 | 97.69 KB | + +With: + +| Method | Mean | Error | StdDev | Allocated | +|-------------- |---------:|----------:|----------:|----------:| +| AllocateArray | 3.489 us | 0.0662 us | 0.0763 us | 97.69 KB | + +Imagine how much time [@YegorStepanov](https://github.com/YegorStepanov) has saved to all the people who so far were removing the columns manually from the results before publishing them on GitHub! + +### Allocation Ratio Column + +In [#1859](https://github.com/dotnet/BenchmarkDotNet/pull/1859) [@YegorStepanov](https://github.com/YegorStepanov) has added Allocation Ratio Column. It's enabled by default when `MemoryDiagnoser` is used and one of the benchmarks is marked as `[Benchmark(Baseline = true)]` or when there are multuple jobs defined and one of them is marked as baseline. + +```cs +[MemoryDiagnoser] +public class AllocationColumnSample +{ + [Benchmark(Baseline = true)] + [Arguments("test")] + public string Builder(string value) + { + StringBuilder sb = new (value); + + for (int i = 0; i < 10; i++) + sb.Append(value); + + return sb.ToString(); + } + + [Benchmark] + [Arguments("test")] + public string Concatenation(string value) + { + string result = value; + + for (int i = 0; i < 10; i++) + result += value; + + return result; + } +} +``` + +| Method | value | Mean | Error | StdDev | Ratio | Gen 0 | Allocated | Alloc Ratio | +|-------------- |------ |---------:|--------:|--------:|------:|-------:|----------:|------------:| +| Builder | test | 127.9 ns | 0.49 ns | 0.43 ns | 1.00 | 0.0544 | 456 B | 1.00 | +| Concatenation | test | 120.2 ns | 0.94 ns | 0.88 ns | 0.94 | 0.0908 | 760 B | 1.67 | + +### Progress and estimated finish time + +In [#1909](https://github.com/dotnet/BenchmarkDotNet/pull/1909) [@adamsitnik](https://github.com/adamsitnik) has added logging of progress and estimated finish time. + +```log +// ** Remained 5211 (99.9%) benchmark(s) to run. Estimated finish 2022-08-25 22:26 (9h 7m from now) ** +``` + +### arm64 support for BenchmarkDotNet.Diagnostics.Windows package + +Due to the [update](https://github.com/dotnet/BenchmarkDotNet/pull/2030) to [TraceEvent 3.0](https://www.nuget.org/packages/Microsoft.Diagnostics.Tracing.TraceEvent) `BenchmarkDotNet.Diagnostics.Windows` package has now arm64 support. Which means that you can use `EtwProfiler` and other ETW-based diagnosers on Windows arm64. + +It would not be possible without [@brianrob](https://github.com/brianrob) who implemented arm64 support for TraceEvent in [#1533](https://github.com/microsoft/perfview/pull/1533)! + +### Hardware Intrinsics information + +In [#2051](https://github.com/dotnet/BenchmarkDotNet/pull/2051) [@adamsitnik](https://github.com/adamsitnik) has extended the hardware information printed in the Summary table with Hardware Intrinsics information. + +![](https://user-images.githubusercontent.com/6011991/180951306-bf8c9f8f-469a-4d7c-8ee1-d7fa47acc0e1.png#mid) + +Sine the space in Summary table is quite limited, we full information is printed only in the log: + +![](https://user-images.githubusercontent.com/6011991/180951531-38c6b6d0-ba6f-4766-a305-b2e306e8420b.png#mid) + +Special thanks to [@tannergooding](https://github.com/tannergooding) who provided a lot of [very valuable feedback](https://github.com/dotnet/BenchmarkDotNet/pull/2051#issuecomment-1194368152) and [@MichalPetryka](https://github.com/MichalPetryka) who contributed an improvement [#2066](https://github.com/dotnet/BenchmarkDotNet/pull/2066) for older runtimes. + +### Other improvements + +* WASM toolchain has received a lot of improvements from various .NET Team members: [#1769](https://github.com/dotnet/BenchmarkDotNet/pull/1769), [#1936](https://github.com/dotnet/BenchmarkDotNet/pull/1936), [#1938](https://github.com/dotnet/BenchmarkDotNet/pull/1938), [#1982](https://github.com/dotnet/BenchmarkDotNet/pull/1982). +* Dependencies and TFMs updates: [#1805](https://github.com/dotnet/BenchmarkDotNet/pull/1805), [#1978](https://github.com/dotnet/BenchmarkDotNet/pull/1978), [#2012](https://github.com/dotnet/BenchmarkDotNet/pull/2012), [#2019](https://github.com/dotnet/BenchmarkDotNet/pull/2019), [#2035](https://github.com/dotnet/BenchmarkDotNet/pull/2035). +* Ensure proper SummaryStyle handling implemented by [@mawosoft](https://github.com/mawosoft) in [#1828](https://github.com/dotnet/BenchmarkDotNet/pull/1828). +* Preserving `EnablePreviewFeatures` project setting which gives the possibility to benchmark preview .NET features. Implemented by [@kkokosa](https://github.com/kkokosa) in [#1842](https://github.com/dotnet/BenchmarkDotNet/pull/1842). +* CI: Using non-deprecated macOS pool on Azure Pipelines, implemented by [@akoeplinger](https://github.com/akoeplinger) in [#1847](https://github.com/dotnet/BenchmarkDotNet/pull/1847) +* CI: Updating Cake to 2.0.0, adopting frosting project style. Implemented by [@AndreyAkinshin](https://github.com/AndreyAkinshin) in [#1865](https://github.com/dotnet/BenchmarkDotNet/pull/1865). +* Detecting ReSharper's Dynamic Program Analysis. Implemented by [@adamsitnik](https://github.com/adamsitnik) in [#1874](https://github.com/dotnet/BenchmarkDotNet/pull/1874). +* Preventing benchmark failure when some of the exporters fail. Implemented by [@epeshk](https://github.com/epeshk) in [#1902](https://github.com/dotnet/BenchmarkDotNet/pull/1902). +* Don't use the diagnosers when benchmarking has failed. Implemented by [@adamsitnik](https://github.com/adamsitnik) in [#1903](https://github.com/dotnet/BenchmarkDotNet/pull/1903). +* Ensuring the default order of benchmarks is the same as declared in source code. Implemented by [@adamsitnik](https://github.com/adamsitnik) in [#1907](https://github.com/dotnet/BenchmarkDotNet/pull/1907). +* Making `BuildTimeout` configurable. Implemented by [@adamsitnik](https://github.com/adamsitnik) in [#1906](https://github.com/dotnet/BenchmarkDotNet/pull/1906). +* Notify users about private methods with Setup/Cleanup attributes. Implemented by [@epeshk](https://github.com/epeshk) in [#1912](https://github.com/dotnet/BenchmarkDotNet/pull/1912). +* Don't run Roslyn Analyzers for the generated code. Implemented by [@adamsitnik](https://github.com/adamsitnik) in [#1917](https://github.com/dotnet/BenchmarkDotNet/pull/1917). +* Ensure `WorkloadActionUnroll` and similar are optimized if possible. Implemented by [@AndyAyersMS](https://github.com/AndyAyersMS) in [#1935](https://github.com/dotnet/BenchmarkDotNet/pull/1935). +* Don't use blocking acknowledgments when there is no need to. Implemented by [@adamsitnik](https://github.com/adamsitnik) in [#1940](https://github.com/dotnet/BenchmarkDotNet/pull/1940). +* Executor: Don't use Process.ExitCode, unless the process has exited. Implemented by [@radical](https://github.com/radical) in [#1947](https://github.com/dotnet/BenchmarkDotNet/pull/1947). +* Revise heuristic for initial jitting. Implemented by [@AndyAyersMS](https://github.com/AndyAyersMS) in [#1949](https://github.com/dotnet/BenchmarkDotNet/pull/1949). +* Allow logging build commands output. Implemented by [@radical](https://github.com/radical) in [#1950](https://github.com/dotnet/BenchmarkDotNet/pull/1950). +* Change Mono AOT mode to Normal AOT with LLVM JIT fall back. Implemented by [@fanyang-mono](https://github.com/fanyang-mono) in [#1990](https://github.com/dotnet/BenchmarkDotNet/pull/1990). + +### Glob filters support for DisassemblyDiagnoser + +So far, the disassembler was always loading the type that was generated by BDN, searching for the benchmark method, disassembling it and when encountered direct method calls, disassembling the called methods as well (if their depth was lesser or equal to max configured depth). + +This was working fine, but only for direct method calls. For indirect, the disassembly was incomplete. + +In [#2072](https://github.com/dotnet/BenchmarkDotNet/pull/2072) [@adamsitnik](https://github.com/adamsitnik) has added the possibility to filter methods disassembled by the `DisassemblyDiagnoser`. + +The users can now pass `--disasmFilter $globPattern` and it's going to be applied to full signatures of all methods available for disassembling. Examples: +* `--disasmFilter *System.Text*` - disassemble all `System.Text` methods. +* `--disasmFilter *` - disassemble all possible methods. + +Moreover, [ClrMD](https://github.com/microsoft/clrmd) was updated to v2 ([#2040](https://github.com/dotnet/BenchmarkDotNet/pull/2040)) and few disassembler bugs have been fixed ([#2075](https://github.com/dotnet/BenchmarkDotNet/pull/2075), [#2078](https://github.com/dotnet/BenchmarkDotNet/pull/2078)). We are expecting that the disassembler will be more reliable now. + +### Docs and Samples improvements + +Big thanks to [@SnakyBeaky](https://github.com/SnakyBeaky), [@Distinctlyminty](https://github.com/Distinctlyminty), [@asaf92](asaf92), [@adamsitnik](https://github.com/adamsitnik) and [@eiriktsarpalis](https://github.com/eiriktsarpalis) who have improved our docs, samples and error messages! + +[#1776](https://github.com/dotnet/BenchmarkDotNet/pull/1776), [#1797](https://github.com/dotnet/BenchmarkDotNet/pull/1797), [#1850](https://github.com/dotnet/BenchmarkDotNet/pull/1850), [#1861](https://github.com/dotnet/BenchmarkDotNet/pull/1861), [#1939](https://github.com/dotnet/BenchmarkDotNet/pull/1939), [#1974](https://github.com/dotnet/BenchmarkDotNet/pull/1974), [#1997](https://github.com/dotnet/BenchmarkDotNet/pull/1997), [#2042](https://github.com/dotnet/BenchmarkDotNet/pull/2042), [#2050](https://github.com/dotnet/BenchmarkDotNet/pull/2050), [#2068](https://github.com/dotnet/BenchmarkDotNet/pull/2068). + +## Bug fixes + +* WASM: [#1811](https://github.com/dotnet/BenchmarkDotNet/pull/1811), [#1846](https://github.com/dotnet/BenchmarkDotNet/pull/1846), [#1916](https://github.com/dotnet/BenchmarkDotNet/pull/1916), [#1926](https://github.com/dotnet/BenchmarkDotNet/pull/1926), [#1932](https://github.com/dotnet/BenchmarkDotNet/pull/1932). +* Diagnoser-provided Analysers weren't automatically added to Config. Fixed by [@mawosoft](https://github.com/mawosoft) in [#1790](https://github.com/dotnet/BenchmarkDotNet/pull/1790). +* Exportes could been duplicated. Fixed by [@workgroupengineering](https://github.com/workgroupengineering) in [#1796](https://github.com/dotnet/BenchmarkDotNet/pull/1796). +* Small bug in SummaryStyle. Fixed by [@mawosoft](https://github.com/mawosoft) in [#1801](https://github.com/dotnet/BenchmarkDotNet/pull/1801). +* `InvalidOperationException/NullReferenceException` in `SmartParaemter`. Fixed by [@mawosoft](https://github.com/mawosoft) in [#1810](https://github.com/dotnet/BenchmarkDotNet/pull/1810). +* Failures caused by colons in benchmark name. Fixed by [@ronbrogan](https://github.com/ronbrogan) in [#1823](https://github.com/dotnet/BenchmarkDotNet/pull/1823). +* Some benchmark arugments were not properly escaped and were causing process launcher to crush. Fixed by [@adamsitnik](https://github.com/adamsitnik) in [#1841](https://github.com/dotnet/BenchmarkDotNet/pull/1841) +* Invalid size specifiers for Memory and Disassembly diagnosers. Fixed by [@YegorStepanov](https://github.com/YegorStepanov) in [#1854](https://github.com/dotnet/BenchmarkDotNet/pull/1854) and [#1855](https://github.com/dotnet/BenchmarkDotNet/pull/1855). +* Respect LogicalGroup order in DefaultOrderer. Fixed by [@AndreyAkinshin](https://github.com/AndreyAkinshin) in [#1866](https://github.com/dotnet/BenchmarkDotNet/pull/1866). +* Endless loop in user interaction with redirected input. Fixed by [@tmds](https://github.com/tmds) in [#](https://github.com/dotnet/BenchmarkDotNet/pull/1870). +* Broken power plan support. Fixed by [@YegorStepanov](https://github.com/YegorStepanov) in [#1885](https://github.com/dotnet/BenchmarkDotNet/pull/1885). +* `BytesAllocatedPerOperation` was not being output by the JSON and XML exporters. Fixed by [#martincostello](https://github.com/martincostello) in [#1919](https://github.com/dotnet/BenchmarkDotNet/pull/1919). +* Incorrect default InvocationCount in the summary table. Fixed by [@AndreyAkinshin](https://github.com/AndreyAkinshin) in [#1929](https://github.com/dotnet/BenchmarkDotNet/issues/1929). +* Failed build output was printed in reverse order. Fixed by [@radical](https://github.com/radical) in [#1945](https://github.com/dotnet/BenchmarkDotNet/pull/1945). +* Build failures due to `NETSDK1150`. Fixed by [@OlegOLK](https://github.com/OlegOLK) in [#1981](https://github.com/dotnet/BenchmarkDotNet/pull/1981). +* `MetricCoumn` was not respecting provided units when formatting values. Fixed by [@mawosoft](https://github.com/mawosoft) in [#2033](https://github.com/dotnet/BenchmarkDotNet/pull/2033). +* Generating invalid code that was causing benchmark failures. Fixed by [@mawosoft](https://github.com/mawosoft) in [#2041](https://github.com/dotnet/BenchmarkDotNet/pull/2041). +* CI: non-master build branches were publishing artifacts to the CI feed. Fixed by [@mawosoft](https://github.com/mawosoft) in [#2047](https://github.com/dotnet/BenchmarkDotNet/pull/2047). +* Comments in the project files were causing build failures. Fixed by [@mawosoft](https://github.com/mawosoft) in [#2056](https://github.com/dotnet/BenchmarkDotNet/pull/2056).