MSTest: add Tests parameter to specify list of tests

[alex-bogomaz]

Hello, please review this PR.
This is a fix for #1601 (MSTest /test support).
Sample usage:

Target "Test" (fun _ ->
    !! (@"UnitTestProject1.dll") 
      |> MSTest (fun p -> { p with Tests = ["UnitTestProject1.TestClass1.TestMethod1"; "TestMethodXX"]})
)

[matthid]

Sorry for the delay. The change looks good, thanks!

Can we migrate the MSTest module to FAKE 5 as suggested in https://fake.build/contributing.html.
There is a guide at the end (https://fake.build/contributing.html#Porting-a-module-to-FAKE-5).

Let me know if you need help.

[alex-bogomaz]

no problem. gimme couple of days

[matthid]

Looks good! I can merge and release this once the build is green. Let me know if there are further bootstrapping problems.

[matthid]

We have a small bootstrapping problem with this ;). I think we can remove those lines for now.

[matthid]

Can we move that up to the other "Modules"? This prevents accidental usage of "old" members (and makes the IDE help you)

[matthid]

OK Travis is still failing:

MSTest runner argument construction, When using Category parameter
» should include the expected category argument (FAIL)
System.TypeInitializationException: The type initializer for '<StartupCode$FakeLib>.$Fake.MSTest' threw an exception. ---> System.Exception: MSTest is not supported on mono platform
  at <StartupCode$FakeLib>.$Fake.MSTest..cctor () [0x00043] in <597af65a7c20dbeda74503835af67a59>:0 
   --- End of inner exception stack trace ---
  at <0x00000 + 0x00000> <unknown method>
  at Test.FAKECore.Testing.MSTestSpecs.BuildArgumentsSpecsBase+<>c.<.ctor>b__5_0 () [0x0000a] in <52dbdee1df254ed68681ec573823124d>:0 
  at (wrapper managed-to-native) System.Reflection.MonoMethod:InternalInvoke (System.Reflection.MonoMethod,object,object[],System.Exception&)
  at System.Reflection.MonoMethod.Invoke (System.Object obj, System.Reflection.BindingFlags invokeAttr, System.Reflection.Binder binder, System.Object[] parameters, System.Globalization.CultureInfo culture) [0x00032] in <a07d6bf484a54da2861691df910339b1>:0 

I think we need to disable the tests on mono.
Now thinking about it, we might need to change the check a bit, because isMono will be false when we switch to dotnetcore as well. So we might need to change that to isUnix.

[alex-bogomaz]

1. can we instead check if current platform is Windows ?

let mstestexe =
   if Environment.isWindows then "mstest.exe" 
   else failwith "MSTest is only supported on Windows platform"

2. what is your preferred way to exclude tests for Mono - use ExcludeTags ?

[matthid]

1. can we instead check if current platform is Windows ?

absolutely :)

2. what is your preferred way to exclude tests for Mono - use ExcludeTags ?

Personally I'm ok with everything that works ... (Don't know MSpec good enough tbh.)

[matthid]

Released in https://www.nuget.org/packages/FAKE/5.0.0-alpha014

[walliski]

Hey! Thanks for fixing this, really looking forward to testing it (pun intended -_-')!

I noted when creating the mstest argument string manually that it is fairly simple to reach the maximum allowed length of windows command line, somewhere around 8000 characters. Our tests can have names closer to 200 chars long. :<

Does FAKE handle this, or is it something that should be included here?

[matthid]

@walliski How could FAKE handle this? To be honest I'm not even sure what would happen if you reach the limit. Maybe windows will strip the commandline or will yield an error?

[walliski]

FAKE could either warn/throw error about it, or then split the execution into multiple parts with a set of tests in each. Alternatively I've understood, though not tested, that you should be able to split the command into multiple lines and this might mitigate the problem. Although this seems to be a bit of a fuzzy area of Windows.

Windows throws an error and wont run the command.