feat(dashboard reporter): Retrieve the project name and version from the project under test using source link #1663
Conversation
Thanks to [SourceLink][1], the repository URL and the full version (including the git SHA1) of a project can be included in the produced assembly. This commit uses the information computed by SourceLink to automatically retrieve the project name (github.com/organization/project) and project version which are required for the dashboard reporter. With this technique, the table on the [Stryker dashboard documentation][2] can be updated with ✅ everywhere for the Stryker.NET column since it doesn't depend on a specific CI provider to provide the repository URL and version. Note that I have run `Stryker.CLI --reporter progress --reporter html --reporter dashboard` on my local checkout of https://github.com/0xced/serilog-formatting-log4net/tree/20e86d55b4bd9e3682c3608530398e794df12481 and it automatically retrieved the project name (`github.com/0xced/serilog-formatting-log4net`) and project version (`1.0.0-rc.2+20e86d55b4bd9e3682c3608530398e794df12481`). The report was successfully uploaded to the [Stryker dashboard][3] [1]: https://github.com/dotnet/sourcelink#readme [2]: https://stryker-mutator.io/docs/General/dashboard/#retrieved-from-environment [3]: https://dashboard.stryker-mutator.io/reports/github.com/0xced/serilog-formatting-log4net/1.0.0-rc.2%2B20e86d55b4bd9e3682c3608530398e794df12481
|
I erroneously closed #1532 by deleting my branch and I was not able to reopen it (the reopen button would stay grayed out). I have now resolved all conflicts and improved the way the project name and version are passed back to the dashboard reporter (no more hacky hidden StrykerOptions references). @rouke-broersma It would be nice to merge this pull request before more conflicts occur. 😉 |
|
Sorry about that! I'll take a look when I get back from holiday tomorrow or Friday. |
|
No problem, enjoy your holiday! 🏖🌴 |
Once stryker-mutator/stryker-net#1663 is merged and Stryker.NET v1.0 is released, the .NET implementation will be able to automatically retrieve the `project` and `version` if the project being mutated is using [SourceLink](https://github.com/dotnet/sourcelink).
|
Done in 7536c9e |
|
/azp run |
|
Azure Pipelines successfully started running 1 pipeline(s). |
src/Stryker.Core/Stryker.Core/Initialisation/InitialisationProcess.cs
Outdated
Show resolved
Hide resolved
rouke-broersma
changed the title
The automatic detection of project name and version introduced in stryker-mutator#1663 broke the DashboardBaselineProvider because the project name would not be set and thus the report would be sent to an invalid URL: `/api/reports//version` This commit addresses this issue by setting the project name and version on the StrykerOptions rather than on the DashboardClient and DashboardReporter (like it was originally done in stryker-mutator#1532).
The automatic detection of project name and version introduced in stryker-mutator#1663 broke the DashboardBaselineProvider because the project name would not be set and thus the report would be sent to an invalid URL: `/api/reports//version` This commit addresses this issue by setting the project name and version on the StrykerOptions rather than on the DashboardClient and DashboardReporter (like it was originally done in stryker-mutator#1532).
* Fix DashboardBaselineProvider The automatic detection of project name and version introduced in #1663 broke the DashboardBaselineProvider because the project name would not be set and thus the report would be sent to an invalid URL: `/api/reports//version` This commit addresses this issue by setting the project name and version on the StrykerOptions rather than on the DashboardClient and DashboardReporter (like it was originally done in #1532). * Fix uploading the report 1. Unlike `GetFromJsonAsync`, `PutAsJsonAsync` does not ensure a success status code so we call it to make sure the request was successful. 2. The description field of a mutant can not be null else we get an HTTP 400 error with this description: > Invalid report. data.files['IndentationSettings.cs'].mutants[0].description should be string * Fix NullReferenceException in InitialisationProcessTests * Don't try to read project name and project version if not required This should fix the integration error on Azure Pipelines: > Failed to retrieve the RepositoryUrl from the AssemblyMetadataAttribute of /home/vsts/work/1/s/integrationtest/TargetProjects/TargetProject/bin/Debug/netcoreapp3.0/TargetProject.dll Co-authored-by: Rouke Broersma <mobrockers@gmail.com>
Once stryker-mutator/stryker-net#1663 is merged and Stryker.NET v1.0 is released, the .NET implementation will be able to automatically retrieve the `project` and `version` if the project being mutated is using [SourceLink](https://github.com/dotnet/sourcelink).
Once stryker-mutator/stryker-net#1663 is merged and Stryker.NET v1.0 is released, the .NET implementation will be able to automatically retrieve the `project` and `version` if the project being mutated is using [SourceLink](https://github.com/dotnet/sourcelink).
Now that stryker-mutator/stryker-net#1663 is merged and Stryker.NET v1.0 is released, the .NET implementation is able to automatically retrieve the `project` and `version` if the project being mutated is using [SourceLink](https://github.com/dotnet/sourcelink).
Now that stryker-mutator/stryker-net#1663 is merged and Stryker.NET v1.0 is released, the .NET implementation is able to automatically retrieve the `project` and `version` if the project being mutated is using [SourceLink](https://github.com/dotnet/sourcelink).
|
@0xced This change causes my project to break with: Is this not be a major breaking change? Or at least have upgrade instructions published somewhere? |
|
This should AFAIK not have been a breaking change. If you used the dashboard reporter previously you would have set the required properties manually and those would still be used. If you turn on the dashboard reporter and don't manually set the required properties Stryker tries to collect them from sourcelink. If you had previously enabled the dashboard reporter not had not set up the required properties I think you would have gotten an error about missing information. Could you open an issue with information about your config that used to work and is no longer working? |
This PR had breaking changes, and I don't have the time to look into it right now: stryker-mutator/stryker-net#1663
|
@AshleighAdams Did you eventually figure it out? When I tried to run Stryker.NET version 1.3.0 with the dashboard reporter on a local checkout of Verlite I encountered no issues. Also, not finding the project version was the consequence of another problem from what I saw in the logs of your GitHub Actions run:
|
|
Hey @0xced, nope I didn't. I dropped the version just as a temporary workaround as I haven't had the time to look into it recently. I also have the same as you though, where it runs with no issue on my local clone. The only differences I can think of are either a partial/shallow clone, the OS it's running on, or something regarding deterministic/reproducible builds being used when doing CI/CD builds (source paths are normalized, which could maybe affect things,). The last one sounds like the most likely candidate to me. |
|
@0xced Yeah, my hunch was correct. Reproducible builds are what break the build. I'll create an issue and add some logs and such. |
|
Issue is here: #1899 |
Thanks to SourceLink, the repository URL and the full version (including the git SHA1) of a project can be included in the produced assembly.
This commit uses the information computed by SourceLink to automatically retrieve the project name (github.com/organization/project) and project version which are required for the dashboard reporter.
With this technique, the table on the Stryker dashboard documentation can be updated with ✅ everywhere for the Stryker.NET column since it doesn't depend on a specific CI provider to provide the repository URL and version.
Note that I have run
Stryker.CLI --reporter progress --reporter html --reporter dashboardon my local checkout of https://github.com/0xced/serilog-formatting-log4net/tree/20e86d55b4bd9e3682c3608530398e794df12481 and it automatically retrieved the project name (github.com/0xced/serilog-formatting-log4net) and project version (1.0.0-rc.2+20e86d55b4bd9e3682c3608530398e794df12481). The report was successfully uploaded to the Stryker dashboard