ArgumentNullException during the mutation test run

[mrlacey]

Describe the bug
A clear and concise description of what the bug is.

Trying Stryker for the first time, but getting an internal exception when generating mutations.

Logs
For us to best assist you in diagnosing the problem we would ask you to run stryker with the option --log-to-file or -L and to attach the resulting logs to the issue.

   _____ _              _               _   _ ______ _______  
  / ____| |            | |             | \ | |  ____|__   __| 
 | (___ | |_ _ __ _   _| | _____ _ __  |  \| | |__     | |    
  \___ \| __| '__| | | | |/ / _ \ '__| | . ` |  __|    | |    
  ____) | |_| |  | |_| |   <  __/ |    | |\  | |____   | |    
 |_____/ \__|_|   \__, |_|\_\___|_| (_)|_| \_|______|  |_|    
                   __/ |                                      
                  |___/                                       


[20:51:27 INF] Logging enabled at level Trace
Version: 4.5.0
[20:51:27 DBG] Stryker starting, version: 4.5.0

[20:51:27 VRB]   CACHE https://api.nuget.org/v3/registration5-gz-semver2/dotnet-stryker/index.json
[20:51:27 VRB]   CACHE https://api.nuget.org/v3/registration5-gz-semver2/dotnet-stryker/index.json
[20:51:27 DBG] Stryker started with options: {"MsBuildPath": null, "DevMode": false, "ProjectPath": "C:\\Users\\matt\\Documents\\GitHub\\mrlacey\\Recurrence", "IsSolutionContext": true, "WorkingDirectory": "C:\\Users\\matt\\Documents\\GitHub\\mrlacey\\Recurrence", "OutputPath": "C:\\Users\\matt\\Documents\\GitHub\\mrlacey\\Recurrence\\StrykerOutput\\2025-02-03.20-51-27", "ReportPath": "C:\\Users\\matt\\Documents\\GitHub\\mrlacey\\Recurrence\\StrykerOutput\\2025-02-03.20-51-27\\reports", "ReportFileName": "mutation-report", "SolutionPath": "C:\\Users\\matt\\Documents\\GitHub\\mrlacey\\Recurrence\\Recurrence.sln", "Configuration": null, "TargetFramework": null, "LogOptions": {"LogToFile": true, "LogLevel": "Verbose", "$type": "LogOptions"}, "MutationLevel": "Standard", "Thresholds": {"High": 80, "Low": 60, "Break": 0, "$type": "Thresholds"}, "AdditionalTimeout": 5000, "LanguageVersion": "Default", "Concurrency": 8, "SourceProjectName": "", "TestProjects": [], "TestCaseFilter": "", "Reporters": ["Progress", "Html"], "WithBaseline": false, "BaselineProvider": "Disk", "AzureFileStorageUrl": "", "AzureFileStorageSas": "", "DashboardUrl": "https://dashboard.stryker-mutator.io", "DashboardApiKey": null, "Since": false, "SinceTarget": "master", "DiffIgnoreChanges": [], "FallbackVersion": "master", "ModuleName": "", "ReportTypeToOpen": null, "Mutate": [{"Glob": {"Tokens": [{"TrailingPathSeparator": {"Value": "\\", "$type": "PathSeparatorToken"}, "LeadingPathSeparator": null, "$type": "WildcardDirectoryToken"}, {"$type": "WildcardToken"}], "$type": "Glob"}, "IsExclude": false, "TextSpans": [{"Start": 0, "End": 2147483647, "Length": 2147483647, "IsEmpty": false, "$type": "TextSpan"}], "$type": "FilePattern"}], "IgnoredMethods": [], "ExcludedMutations": [], "ExcludedLinqExpressions": [], "OptimizationMode": "CoverageBasedTest", "ProjectName": "", "ProjectVersion": "", "BreakOnInitialTestFailure": false, "MutantIdProvider": {"$type": "BasicIdProvider"}, "$type": "StrykerOptions"}
[20:51:27 INF] Analysis starting.
[20:51:27 INF] Identifying projects to mutate in C:\Users\matt\Documents\GitHub\mrlacey\Recurrence\Recurrence.sln. This can take a while.
[20:51:27 DBG] Analyzing 2 projects.
[20:51:27 DBG] Analyzing Recurrence\Recurrence.csproj
[20:51:27 DBG] Analyzing Recurrence.Tests\Recurrence.Tests.csproj
[20:51:38 DBG] Analysis of project Recurrence.Tests\Recurrence.Tests.csproj succeeded.
[20:51:56 INF] Analysis complete.
[20:51:56 ERR] An error occurred during the mutation test run 
System.ArgumentNullException: Value cannot be null. (Parameter 'folderName')
   at Stryker.Core.Initialisation.InputFileResolver.AnalyzeAllNeededProjects(List`1 projectList, IStrykerOptions options, IAnalyzerManager manager, ScanMode mode) in /_/src/Stryker.Core/Stryker.Core/Initialisation/InputFileResolver.cs:line 175
   at Stryker.Core.Initialisation.InputFileResolver.AnalyzeAndIdentifyProjects(List`1 projectList, IStrykerOptions options, IAnalyzerManager manager, ScanMode mode) in /_/src/Stryker.Core/Stryker.Core/Initialisation/InputFileResolver.cs:line 124
   at Stryker.Core.Initialisation.InputFileResolver.ResolveSourceProjectInfos(IStrykerOptions options) in /_/src/Stryker.Core/Stryker.Core/Initialisation/InputFileResolver.cs:line 66
   at Stryker.Core.Initialisation.InitialisationProcess.GetMutableProjectsInfo(IStrykerOptions options) in /_/src/Stryker.Core/Stryker.Core/Initialisation/InitialisationProcess.cs:line 56
   at Stryker.Core.Initialisation.ProjectOrchestrator.MutateProjects(IStrykerOptions options, IReporter reporters, ITestRunner runner) in /_/src/Stryker.Core/Stryker.Core/Initialisation/ProjectOrchestrator.cs:line 53
   at Stryker.Core.StrykerRunner.RunMutationTest(IStrykerInputs inputs, ILoggerFactory loggerFactory, IProjectOrchestrator projectOrchestrator) in /_/src/Stryker.Core/Stryker.Core/StrykerRunner.cs:line 65
[20:51:56 INF] Time Elapsed 00:00:29.4026273

Expected behavior
A clear and concise description of what you expected to happen.

Not throw an error and test some mutations.

Desktop (please complete the following information):

• OS: Windows
• Type of project: MAUI (& MSTest)
• Framework Version: net9.0
• Stryker Version: 4.5.0

Additional context
Add any other context about the problem here.

Not sure what else you want/need to know.

[dupdob]

Hi,
thanks for reporting this. Could you share the related project?
I am afraid we will have a hard time figuring out what goes wrong without it.
If you have confidentiality issues with it, we just need the solution file and the csproj ones. And you can share them privately (via the Stryker's Slack DMs or via email).

[mrlacey]

@dupdob I've created a minimal project that reproduces this at https://github.com/mrlacey/StrykerBugRepro

I noticed that I was trying to run with the working directory of the solution file, but even when I set the working directory to the unit test project directory I still get the same exception.

[dupdob]

Thanks for the repro project.
The project analysis step fails, but the failure is not properly handled and is surfaced via some unrelated exception.
This part will be fixed (ensuring a correct error message).
But I am afraid the underlying problem may be more difficult to fix. We use 'Buildalyzer' to analyze project and extract needed information, hence the problem lies within this component, which is another OSS project.

As of now, we do not properly support MAUI project due to problem with Buildalyzer.

[mrlacey]

As of now, we do not properly support MAUI project due to problem with Buildalyzer.

Good to know. Thank you.

[dupdob]

Update: I fixed the undue exception. Then, I got the correct error: the problem was that the project could not compile on my machine due to missing SDKs.
I am adding these SDKs and testing again.

ReUpdate:
I have now another problem as Stryker fails to identify the relationship between the test project and the project to mutate.
This is due to the TargetRefPath MsBuild properties containing some made up folder instead of the real one.

[dupdob]

Update: I added a workaround to the previous issue; so I get to the point where Stryker tries to mutate the project but then fails because it can't build it.
It looks like it is a problem with global usings: those are supposed to end in a generated cs file, but the file is not actually generated, so the build fails as the needed using are not there....

[dupdob]

I opened PR #3184 that improves the situation. But it does not fully work on your sample projects because initial test phase fails.
So I do not consider this issue as closed

[Aakashdeepcodes]

[20:51:27 INF] Logging enabled at level Trace
Version: 4.5.0
[20:51:27 DBG] Stryker starting, version: 4.5.0

[20:51:27 DBG] Stryker started with options:
{"ProjectPath": "C:\Users\matt\Documents\GitHub\mrlacey\Recurrence", "IsSolutionContext": true, "WorkingDirectory": "C:\Users\matt\Documents\GitHub\mrlacey\Recurrence", "OutputPath": "C:\Users\matt\Documents\GitHub\mrlacey\Recurrence\StrykerOutput\2025-02-03.20-51-27", "SolutionPath": "C:\Users\matt\Documents\GitHub\mrlacey\Recurrence\Recurrence.sln", "LogOptions": {"LogToFile": true, "LogLevel": "Verbose"}, "MutationLevel": "Standard", "Concurrency": 8, "Reporters": ["Progress", "Html"]}

[20:51:27 INF] Analysis starting.
[20:51:27 INF] Identifying projects to mutate in C:\Users\matt\Documents\GitHub\mrlacey\Recurrence\Recurrence.sln
[20:51:27 DBG] Analyzing 2 projects: Recurrence.csproj and Recurrence.Tests.csproj
[20:51:56 ERR] An error occurred during the mutation test run

System.ArgumentNullException: Value cannot be null. (Parameter 'folderName')
at Stryker.Core.Initialisation.InputFileResolver.AnalyzeAllNeededProjects()
at Stryker.Core.Initialisation.InputFileResolver.ResolveSourceProjectInfos()
at Stryker.Core.StrykerRunner.RunMutationTest()

[20:51:56 INF] Time Elapsed 00:00:29.4026273