Support .net6 runtime assemblies, new Build system, and Legacy Fake Codebase Separation #2609
Conversation
|
Chiming in to say a giant +1 on deleting the fake4 codebase at this time. |
|
Great to see the work in progress! |
yazeedobaid
force-pushed
the
net6-runtime-reference-assemblies
branch
2 times, most recently
from
c250b10 to
fbadd28
Compare
yazeedobaid
changed the title
yazeedobaid
force-pushed
the
net6-runtime-reference-assemblies
branch
from
fbadd28 to
5981ba0
Compare
…paration of legacy codebase.
yazeedobaid
force-pushed
the
net6-runtime-reference-assemblies
branch
from
b5ae18c to
247209c
Compare
| // please see: https://docs.microsoft.com/en-us/dotnet/core/rid-catalog | ||
| // some native libraries, such as Sqlite.Interop are distributed without version specific. | ||
| // THe runtime identifier on windows 10 machine will be win10-x64 however, Sqlite.Interop | ||
| // will have an RID of win-x64 | ||
| let runtimeLibrariesNotVersionSpecific = | ||
| installModel.GetRuntimeLibraries graph ridNotVersionSpecific targetProfile | ||
| |> Seq.map (fun fi -> DependencyFile.Library { File = fi.Library.Path }) | ||
| |> Seq.toList | ||
|
|
There was a problem hiding this comment.
I'm not sure if this is how it is supposed to work, I'd assume some libraries might still run into issues.
Basically the runtime identifiers are organized like a tree (https://docs.microsoft.com/en-us/dotnet/core/rid-catalog), last time I checked those RIDs where pushed with a NuGet package (you should find the logic in the Paket.Core code). This might have changed by now.
But basically we should know that a win-x64 is compatible on a win10-x64 runtime and take that (disclaimer: I wrote that code in Paket and it might have changed by now or is no longer working)
Of course we can release it as-is and see if someone complains.
There was a problem hiding this comment.
For the native assemblies not sure why it breaks, but the two changes I think affect it are:
- Either
Paket.Core, in which we updated to a major version, so it may have some breaking changes in that area. - Or the removal of
Microsoft.DotNet.PlatformAbstractionssince it is now deprecated by Microsoft and using built-in runtime identifier property.
As you pointed out it should resolve the native package since it is a graph and win10-x64 is a child of win-x64. When I debug it, the InstallModel resolved it with win10-x64 but the runtime identifier from SDK was win-x64, and InstallModel didn't return anything in that configuration.
I'm not familiar with this area, if you have any suggestions I'm all open.
There was a problem hiding this comment.
If you want to take a look, I'd say that if you debugged it to InstallModel it is most likely a Paket bug, so the easiest would be to add a unit-test there and try to solve it there and then ask @forki to make a release :)
IIRC there are already tests for the runtime identifiers there, so it shouldn't be too hard to add this scenario (but that is always easy to write, I could easily be wrong on that)
|
@yazeedobaid This is amazing work! Do you need further review before merging? |
|
@yazeedobaid I'd love to see this merged, I think I just hit a need for net6 support. Can we proceed? Thanks |
|
@matthid Thanks for the review! So I renamed the class to |
@dsyme Yes, we can! I'm done from my side. I think we can go with an alpha release - 5.20.5-alpha. |
yazeedobaid
force-pushed
the
net6-runtime-reference-assemblies
branch
from
6a6e44a to
9f97e0d
Compare
|
@yazeedobaid Please check again, I assigned you the admin role. |
|
Thanks @matthid |
|
You guys rock. Really impressive to see you deliver this |
…odebase Separation (#2609) * .net6 runtime assemblies in Fake runner, using GitHub actions, and separation of legacy codebase. * naming convention changes * disabiling stale bot
…odebase Separation (#2609) * .net6 runtime assemblies in Fake runner, using GitHub actions, and separation of legacy codebase. * naming convention changes * disabiling stale bot
…odebase Separation (#2609) * .net6 runtime assemblies in Fake runner, using GitHub actions, and separation of legacy codebase. * naming convention changes * disabiling stale bot
…odebase Separation (fsprojects#2609) * .net6 runtime assemblies in Fake runner, using GitHub actions, and separation of legacy codebase. * naming convention changes * disabiling stale bot
Description
Introduction
This PR introduce the following modifications to Fake codebase and the build system:
build.fsxFileEach point is explained in the following sections.
.Net6 Runtime Assemblies Support
This PR adds support for .NET 6 runtime assemblies in Fake runner, reported in issue #2588. As pointed out in the issue discussion, Fake runner pins its execution to NETSTANDARD2.0, so adding a dependency that supports .NET5 or .NET6 only will fail. The move from NETSTANDARD2.0 to .NET6 is not straightforward, since in NETSTANDARD2.0 runtime assemblies that are needed by the script are installed from a NuGet package called
NETStandard.Library. However, in .NET5 and .NET6 it is not the same.To support new runtime assemblies, this PR make changes to
Fake.Runtimeproject, the changes introduce a pinned framework in which Fake runner will use always in resolving dependencies and framework restrictions. Since .NET6 is the next LTS release, it was used instead of Netstandard2.0 which was used previously. Secondly, it will try to resolve the runtime assemblies as follows:global.jsonfile from script directory. If it find one it will checks the SDK version in it.NETStandard.LibraryNuGet package.So, to use .NET6 runtime assemblies, a
global.jsonfile must exists with SDK version set to .NET6. Otherwise, it will default to using Netstandard2.0 assemblies.For the case reported in 2588 issue, the following gist provide a simple script that demonstrate how to use .NET6 runtime assemblies by including a
global.jsonfile with .NET6 SDK specified.The benefits of this solution is the switch between runtime assemblies will now be much easier. For example, when .NET6 is officially released and adopted more. We can drop .Netstandard2.0 and use by default .NET6 - for that Fake runner will require .NET6 as a requirement - and provide a switch for next release of .NET. For example when .NET7 is released, runtime assemblies for it can be activated with a
global.jsonfile.The cons of this solution as pointed out in the issue is that, Fake runner now depends on an installed SDK.
In addition to runtime assemblies reference changes. A change was also introduce to the target profile argument in FSI. In FSI, the target profile for running an interactive script is different between .NET and .Netstandard. So a switch was included to handle that by resolved assemblies version in
CompileRunner.fs.I added two tests in integration tests project to test running Fake runner with .Netstandard2.0 and .NET6 runtime assemblies in
Fake.DotNet.RuntimeReferenceAssemblies.fs→ This handle issue #2588
Legacy Fake Codebase Separation
During the implementation, I faced an issue with dependencies. I wasn't able to detect what was happening and the solution was to remove all the references in the main group in
paket.dependeciesfile which as I found is for the legacy Fake build. I also disabled the build and release for legacy Fake in the Azure pipeline using the environment variable to disable legacy build. My argument for this is we need to move on with the Fake 5 implementation.Also, while doing that, I moved the current
paket.dependenciesandpaket.lockfiles to legacy directory and I marked them as ignored for reference. In addition, I moved other files/directories to legacy directory as a form of separation between Fake 5 codebase and legacy Fake codebase, which are:paket.dependencies→src/legacy/[ignored]paket.dependenciespaket.lock→src/legacy/[ignored]paket.locklegacy-build.fsx→src/legacy/legacy-build.fsx.gitlab-ci.yml→src/legacy/[ignored].gitlab-ci.yml.travis.yml→src/legacy/[ignored].travis.ymlappveyor.yml→src/legacy/[ignored]appveyor.ymlLegacy-FAKE.sln→src/legacy/Legacy-FAKE.slnLegacy-FAKE.sln.DotSettings→src/legacy/Legacy-FAKE.sln.DotSettingsSamples/→src/legacy/Samples/Vagrantfile→src/legacy/Vagrantfilebuild-web-bundles.cmd→src/legacy/[ignored]build-web-bundles.cmdbuild-web-bundles.fsx→src/legacy/[ignored]build-web-bundles.fsxAlso, the following Paket files have been marked as ignored to not picked up by Paket:
src/legacy/FAKE/paket.references→src/legacy/FAKE/[ignored]paket.referencessrc/legacy/Fake.Experimental/ignored]paket.referencessrc/legacy/Fake.FluentMigrator/ignored]paket.referencessrc/legacy/Fake.Gallio/ignored]paket.referencessrc/legacy/Fake.IIS/ignored]paket.referencessrc/legacy/Fake.SQL/ignored]paket.referencessrc/legacy/FakeLib/ignored]paket.referencessrc/legacy/FsCheck.Fake/ignored]paket.referencessrc/legacy/Test.FAKECore/ignored]paket.referencessrc/legacy/Test.Git/ignored]paket.referencesBy these changes and changes to
build.fsxscript, legacy Fake release will be discontinued.Using GitHub Actions for Build and Test
This PR also adds two GitHub actions for build, test, and preparing for a release. Each one is explained as follows:
The
build_and_testaction builds the repository from PRs and Pushes torelease/nextbranch on three OSs which simulate Appveyor, Travis, and Azure pipelines. It call the build script with theDefaulttarget which build the project and run the tests.The steps before calling the build script pack the Fake runner and replace the current
fake-cliDotNet tool in it with produced version. I did this for two reasons:Paket.Coreversion between runner versions will not enable build system to succeed. As is happening currently with the Azure pipeline. It fails due to changes inpaket.lockfile which was produced by Paket with version6.0.13which adds target frameworks for .Net6. And previous Fake runner uses older version of Paket which don't understand .Net6 target frameworks.The
prepare_releaseaction is intended to be run on pushes onrelease/nextand/ormainbranches for preparing artifacts for a release using Windows OS which simulates current Azure CI pipeline. It uses similar steps asbuild_and_testaction except it run build script withRelease_BuildAndTesttarget to produce artifacts and collect them.Not sure if this actions should be run on
release/nextormainbranches?! If we run it onrelease/nextwe will get artifacts published for each change. So this will enable feedback and testing more frequent. Onmainbranch, it will be when a merge fromrelease/nexttomainis done for releasing, which is less frequent. Currently I set the trigger to pushes onrelease/nextbranch.These two actions can be used as in-house build and test for Fake. For release, we can still use current Azure release pipeline but give it the artifacts from
prepare_releaseaction.The following run has the artifacts produced from
prepare_releaseaction - at the bottom of the page.→ This handles issue #2607
Modifications to
build.fsxFileI have tried to organize the build script a little pit and while separating the legacy build. These include:
#if BOOTSTRAPdirective has been removed since GitHub actions uses caching step which caches dependencies. And while working on this PR, changes to build script dependencies didn't effect until I removed it..fake/build.fsx/intellisense.fsxfile and call to load it in build script has been removed since it was needed for legacy build.publishmethod now copy build artifacts to a directory to be picked up by release action.--no-restoreoption since it seems .NET6 SDK is not supported yet in Debian distribution as pointed out here. Without this option, thedotnet debcommand will try to use .NET5 references and will fail since they are not installed in build server and we use .NET6.RunIdenvironment variable and commit SHA as a metadata for version.Other Modifications
Microsoft.DotNet.PlatformAbstractionshas been removed since it is deprecated in .NET6 as pointed out hereUnable to load DLL 'SQLite.Interop.dll': The specified module could not be found..TheSQLite.Interoplibrary uses runtime identifier without a version specific. On windows for example, it was win-x64, runtime graph uses version specific runtime identifier, on my machine it was win10-x64. I referred to this documentation page from Microsoft.ToDos
If this PR is accepted the following ToDos are still needed to be done:
Summary
Sorry for making this PR very long and full of modifications, but all issues handled in this PR are related to each other in some way. I tried to document every modification I did in it. Please have a look at the proposal for supporting .NET6 runtime assemblies, new build system, and modification to separate legacy codebase from Fake 5 codebase. Your feedback is highly appreciated.
Thanks
TODO
Feel free to open the PR and ask for help
New (API-)documentation for new features exist (Note: API-docs are enough, additional docs are in
help/markdown)unit or integration test exists (or short reasoning why it doesn't make sense)
boy scout rule: "leave the code behind in a better state than you found it" (fix warnings, obsolete members or code-style in the places you worked in)
(if new module) the module has been linked from the "Modules" menu, edit
help/templates/template.cshtml, linking to the API-reference is fine.(if new module) the module is in the correct namespace
(if new module) the module is added to Fake.sln (
dotnet sln Fake.sln add src/app/Fake.*/Fake.*.fsproj)Fake 5 API guideline is honored