Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[nnyeah] Start testing #14913

Merged
merged 25 commits into from
May 9, 2022
Merged

[nnyeah] Start testing #14913

merged 25 commits into from
May 9, 2022

Conversation

stephen-hawley
Copy link
Contributor

Added code to:

  • compile a string to a platform library
  • collect the output of the compilation process
  • check for errors

Added a single unit test of the smoke test variety.

This code is currently missing the process of running nnyeah on the output and compiling and running a .NET app and collecting the output.

@stephen-hawley stephen-hawley added enhancement The issue or pull request is an enhancement skip-all-tests Skip all the tests not-notes-worthy Ignore for release notes labels May 5, 2022
Copy link
Member

@dalexsoto dalexsoto left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's discuss tomorrow, I think we can do Roslyn now if we really want to go down this path but I think we could discuss different alternatives.

@vs-mobiletools-engineering-service2

This comment has been minimized.

@vs-mobiletools-engineering-service2

This comment has been minimized.

Copy link
Member

@mandel-macaque mandel-macaque left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This won't compile because it is not using dotnet 6 and the bots do not install .net framework

Copy link
Contributor

@chamons chamons left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good enough to get keep momentum going. We can rework later if needed.

(Once we fix the build error obviously)

if (output is null)
output = new StringBuilder ();

if (env != null) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Lets be consistent

Suggested change
if (env != null) {
if (env is not null) {

Comment on lines 12 to 13
if (callingMethod is null)
Assert.Fail ("unable to get calling test from stack frame.");
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can do:

Suggested change
if (callingMethod is null)
Assert.Fail ("unable to get calling test from stack frame.");
Assert.NotNull (callingMethod, "unable to get calling test from stack frame.);

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We could even ut an is and remove all those ! everywhere

public static void TestAndExecute (string libraryCode, string callingCode, string expectedOutput,
string callingMethodName = "")
{
var testName = GetInvokingTestName (out var nameSpace, callingMethodName);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nameSpace is never used.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I assume that's for use later, this method is not finished.

string callingMethodName = "")
{
var testName = GetInvokingTestName (out var nameSpace, callingMethodName);
var testClassName = "NnyeahTest" + testName;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same never used.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ditto.

using var initialLibraryDir = new DisposableTempDirectory ();
using var finalLibraryDir = new DisposableTempDirectory ();

var libCompilerOutput = BuildLibrary (libraryCode, testName, initialLibraryDir.DirectoryPath);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same, never used.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ditto.

return callingMethodName;
}

public static void TestAndExecute (string libraryCode, string callingCode, string expectedOutput,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

callingCode and expectedOutput are not used.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ditto. I'm just going to leave these ununsed bits in case we wire in test execution.

@vs-mobiletools-engineering-service2

This comment has been minimized.

@@ -4,6 +4,8 @@
using System.Runtime.InteropServices;
using System.Threading;

#nullable disable
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll take care of this in a diff PR.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@@ -16,6 +16,8 @@
using System.Threading;
using System.Threading.Tasks;

#nullable disable
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll take care of this in a diff PR.

@@ -3,6 +3,8 @@
using System.Linq;
using System.Text;

#nullable disable
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll take care of this in a diff PR.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done here #14944

{
var compilerArgs = BuildCompilerArgs (sourceFiles, outputFile, platformName, isLibrary);
Execution execution = await Execution.RunAsync(MonoCompiler, compilerArgs, mergeOutput: true, workingDirectory: workingDirectory);
return execution!.StandardOutput.ToString()!;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll fix this need of ! in a follow up PR.

@vs-mobiletools-engineering-service2

This comment has been minimized.

@vs-mobiletools-engineering-service2

This comment has been minimized.

@vs-mobiletools-engineering-service2

This comment has been minimized.

@vs-mobiletools-engineering-service2
Copy link
Collaborator

📋 [PR Build] API Diff 📋

API Current PR diff

✅ API Diff (from PR only) (no change)

View API diff
View dotnet API diff
View dotnet legacy API diff
View dotnet iOS-MacCatalayst API diff

API diff

✅ API Diff from stable

View API diff
View dotnet API diff
View dotnet legacy API diff
View dotnet iOS-MacCatalayst API diff

Generator diff

Generator Diff (no change)

Pipeline on Agent XAMBOT-1096.Monterey
Hash: 20ebca72790fa9146ddb5dc7d56e7c2d827d73d5

@vs-mobiletools-engineering-service2
Copy link
Collaborator

📚 [PR Build] Artifacts 📚

Packages generated

View packages

Pipeline on Agent XAMBOT-1102.Monterey'
Hash: 20ebca72790fa9146ddb5dc7d56e7c2d827d73d5

@vs-mobiletools-engineering-service2
Copy link
Collaborator

❌ [PR Build] Tests on macOS Mac Catalina (10.15) failed ❌

Failed tests are:

  • linksdk
  • linkall

Pipeline on Agent
Hash: 20ebca72790fa9146ddb5dc7d56e7c2d827d73d5

@vs-mobiletools-engineering-service2
Copy link
Collaborator

❌ [PR Build] Tests on macOS M1 - Mac Big Sur (11.5) failed ❌

Failed tests are:

  • linksdk
  • linkall
  • xammac_tests
  • monotouch-test

Pipeline on Agent
Hash: 20ebca72790fa9146ddb5dc7d56e7c2d827d73d5

@vs-mobiletools-engineering-service2
Copy link
Collaborator

✅ [CI Build] Tests passed on VSTS: simulator tests iOS. ✅

Tests passed on VSTS: simulator tests iOS.

🎉 All 2 tests passed 🎉

Pipeline on Agent XAMBOT-1042.Monterey'
Merge 20ebca7 into a9cab8f

@mandel-macaque
Copy link
Member

Failures are unrelated and they are network based.

@stephen-hawley stephen-hawley merged commit 5631ca5 into dotnet:main May 9, 2022
@stephen-hawley stephen-hawley deleted the StartTesting branch May 9, 2022 13:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement The issue or pull request is an enhancement not-notes-worthy Ignore for release notes skip-all-tests Skip all the tests
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants