Improving Logic To Pass Sources To Adapters

[abhishkk]

Issues with proposed changes

1. Using FileExtension attribute for specifying native and managed support is not correct. We are using this attribute to allow adapter to not break compat and still use this feature.
2. We are saving only the adapter discovery time and not adapter load time. Even if test source assembly is native, we will load all adapters passed (including managed adapters). But discovery will not be done by managed adapters for this native assembly.
3. Its not job of test host process to differentiate between native and managed assembly. Ideally there should have been separate testhost process for native assemblies (similar to what we have for dotnet core).

Questions

Question 1: Should we skip loading of adapters in InitializeExtensions call in test host. We can just create lazy extensions in InitializeExtensions call. In DiscoverTests/RunTests call we can load only those adapters which have some matching sources.

Alternative

Alternative 1: Spawning separate test host process for native and managed assemblies
We can do the matching of source based on file extensions and assembly type on vstest.console side. If both managed and native assemblies are found, we can spawn up two separate test host process. One to discover/run native assemblies and another to run managed assemblies.

Issues:

1. We need to read adapters at vstest.console side. Currently its done by test host only.
2. A lot of changes are required to support this scenario.

Alternative 2: Creating testhost native process to run/discover native assemblies

Issues:

1. We need to create native adapter and native test host process for this which requires lot of changes.

[pvlakshm]

Was just looking at the second update ....
This looks like a breaking change.
What happens if there is a C++ adapter that did not specify this?
What if I am on the latest version of the TP (with the changes that look for the category attribute, but using an older version of a C++ test adapter?
What happens in the case of a team setting where one of the devs might be on VS15.Latest, but another dev is on VS15.5?

[abhishkk]

@pvlakshm I have updated the RFC with changes where we are not considering the adapter managed by default. Can you review?