dotnet publish from non-Windows does not copy required native files

[dtchepak]

When publishing an app from Mac targeting Windows, required native files are not included. The same app built from Mac using dotnet build works fine on both platforms.

I'm not sure if this is me misunderstanding dotnet publish, a problem with my configuration, or a potential problem with the CLI. Appreciate any help you can provide. :)

Steps to reproduce

Create CblCount.csproj:

<Project Sdk="Microsoft.NET.Sdk">
  <PropertyGroup>
    <OutputType>Exe</OutputType>
    <TargetFramework>netcoreapp3.0</TargetFramework>
  </PropertyGroup>
  <ItemGroup>
    <PackageReference Include="Couchbase.Lite" Version="2.6.0" />
    <PackageReference Include="Couchbase.Lite.Support.NetDesktop" Version="2.6.0" />
  </ItemGroup>
</Project>

Create Program.cs:

using System;
using System.IO;
using Couchbase.Lite;

namespace CblCount {
    class Program {
        private const string Cblite2Suffix = ".cblite2";

        static void Main(string[] args) {
            if (args.Length != 1) {
                Console.WriteLine("Usage: CblCount (path_to_cblite2_db)");
                return;
            }
            var dbPath = args[0];
            if (!Directory.Exists(dbPath)) {
                Console.WriteLine($"{dbPath} does not exist");
                return;
            } else if (!dbPath.EndsWith(Cblite2Suffix)) {
                Console.WriteLine($"{dbPath} must end with {Cblite2Suffix}");
                return;
            }
            // CBL .NET expects path without .cblite2 suffix and without trailing path separator
            var pathWithoutSuffix = dbPath.Substring(0, dbPath.Length - Cblite2Suffix.Length);

            var db = new Database(pathWithoutSuffix);
            var count = db.Count;
            var countString = count == 1 ? "1 document" : $"{count} documents";
            Console.WriteLine($"{dbPath} contains {countString}.");
        }
    }
}

Create a fake Couchbase database:

$ mkdir db.cblite2

Can then run on any platform using:

~/dev/git/CblCount  % dotnet run db.cblite2
db.cblite2 contains 0 documents.

The output directory includes native Couchbase components:

~/dev/git/CblCount  % ls -R bin/Debug/netcoreapp3.0/runtimes/*/native
bin/Debug/netcoreapp3.0/runtimes/linux-x64/native:
libLiteCore.so   libc++.so.1      libc++abi.so.1   libicudata.so.54 libicui18n.so.54 libicuuc.so.54

bin/Debug/netcoreapp3.0/runtimes/osx-x64/native:
libLiteCore.dylib

bin/Debug/netcoreapp3.0/runtimes/win7-x64/native:
LiteCore.dll LiteCore.pdb

bin/Debug/netcoreapp3.0/runtimes/win7-x86/native:
LiteCore.dll LiteCore.pdb

Now publish:

$ dotnet publish -r win-x64 --self-contained true CblCount.csproj -o publish/win
$ dotnet publish -r osx-x64 --self-contained true CblCount.csproj -o publish/mac

Observed results

The Mac app works as expected:

~/dev/git/CblCount  % ./publish/mac/CblCount db.cblite2
db.cblite2 contains 0 documents.

Copying publish/win to Desktop\win on Windows and then running gives the following error (with paths elided):

$ ./CblCount.exe test.cblite2
Unhandled exception. System.TypeInitializationException: The type initializer for 'Couchbase.Lite.Database' threw an exception.
 ---> System.TypeInitializationException: The type initializer for 'LiteCore.Interop.Native' threw an exception.
 ---> System.DllNotFoundException: Could not find LiteCore.dll!  Nothing is going to work!
Tried searching in:
...\runtimes\win7-x64\native\LiteCore.dll
...\Desktop\win\x64\LiteCore.dll
...\Desktop\win\bin\x64\LiteCore.dll

   at Couchbase.Lite.Support.NetDesktop.LoadLiteCore()
   at LiteCore.Interop.Native..cctor()
   --- End of inner exception stack trace ---
   at LiteCore.Interop.Native.c4log_getDomain(Byte* name, Boolean create)
   at Couchbase.Lite.Logging.FileLogger.SetupDomainObjects()
   at Couchbase.Lite.Logging.FileLogger..ctor()
   at Couchbase.Lite.Logging.Log..ctor()
   at Couchbase.Lite.Database..cctor()
   --- End of inner exception stack trace ---
   at Couchbase.Lite.Database..ctor(String name, DatabaseConfiguration configuration)
   at CblCount.Program.Main(String[] args) in .../CblCount/Program.cs:line 29

Inspecting the published output shows that x64/LiteCore.dll and x86/LiteCore.dll are missing (these are in build output under bin/Debug/netcoreapp3.0/runtimes/win7-*/native, just not in published output).

If I run the dotnet publish -r win-x64 command from the Windows machine, the application works as expected:

$ dotnet publish -r win-x64 -o pub/win --self-contained true
Microsoft (R) Build Engine version 16.3.0+0f4c62fea for .NET Core
Copyright (C) Microsoft Corporation. All rights reserved.

  Restore completed in 55.13 ms for ...\Desktop\CblCount\CblCount.csproj.
  CblCount -> ...\Desktop\CblCount\bin\Debug\netcoreapp3.0\win-x64\CblCount.dll
  CblCount -> ...\Desktop\CblCount\pub\win\

$ ./pub/win/CblCount.exe db.cblite2
db.cblite2 contains 0 documents.

The app built on Windows targeting osx-x64 also works correctly once the appropriate chmod u+x permission update has been applied.

Expected results

I expected that publishing on any platform and targeting using -r would include the required native files for that platform. In my actual use case I am building on Linux using Docker and trying to get Mac and Windows apps from that CI build.

Other things tried

I have tried different RIDs (both in publish command and in .csproj), different settings for --self-contained, and also adding <IncludeSymbolsInSingleFile>true</IncludeSymbolsInSingleFile> to csproj as I saw suggested in another issue. I also get the same results using /p:PublishSingleFile=true (which is what I'm ultimately aiming to get for the Windows app).

I have not tried updating csproj to manually copy over these files based on targeted RID yet, or specifying different publish options.

Possibly related

• https://github.com/dotnet/cli/issues/12981
• https://github.com/dotnet/cli/issues/4481
• https://github.com/dotnet/cli/issues/12723
• csproj: document how to properly pack platform-specific native assemblies NuGet/Home#6645

Details

Mac info

% dotnet --info
.NET Core SDK (reflecting any global.json):
 Version:   3.0.100
 Commit:    04339c3a26

Runtime Environment:
 OS Name:     Mac OS X
 OS Version:  10.15
 OS Platform: Darwin
 RID:         osx.10.15-x64
 Base Path:   /usr/local/share/dotnet/sdk/3.0.100/

Host (useful for support):
  Version: 3.1.0-preview1.19506.1
  Commit:  bbf5542781

.NET Core SDKs installed:
  1.0.4 [/usr/local/share/dotnet/sdk]
  2.0.0 [/usr/local/share/dotnet/sdk]
  2.1.4 [/usr/local/share/dotnet/sdk]
  2.1.301 [/usr/local/share/dotnet/sdk]
  2.1.302 [/usr/local/share/dotnet/sdk]
  2.1.503 [/usr/local/share/dotnet/sdk]
  2.1.504 [/usr/local/share/dotnet/sdk]
  2.1.505 [/usr/local/share/dotnet/sdk]
  2.2.105 [/usr/local/share/dotnet/sdk]
  2.2.203 [/usr/local/share/dotnet/sdk]
  3.0.100 [/usr/local/share/dotnet/sdk]

.NET Core runtimes installed:
  Microsoft.AspNetCore.All 2.1.1 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.All]
  Microsoft.AspNetCore.All 2.1.2 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.All]
  Microsoft.AspNetCore.All 2.1.7 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.All]
  Microsoft.AspNetCore.All 2.1.8 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.All]
  Microsoft.AspNetCore.All 2.1.9 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.All]
  Microsoft.AspNetCore.All 2.2.3 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.All]
  Microsoft.AspNetCore.All 2.2.4 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.All]
  Microsoft.AspNetCore.App 2.1.1 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 2.1.2 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 2.1.7 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 2.1.8 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 2.1.9 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 2.2.3 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 2.2.4 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 3.0.0-preview8.19405.7 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 3.0.0-rc1.19457.4 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 3.0.0 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 3.1.0-preview1.19508.20 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.App]
  Microsoft.NETCore.App 1.0.0-rc2-3002702 [/usr/local/share/dotnet/shared/Microsoft.NETCore.App]
  Microsoft.NETCore.App 1.0.0 [/usr/local/share/dotnet/shared/Microsoft.NETCore.App]
  Microsoft.NETCore.App 1.0.5 [/usr/local/share/dotnet/shared/Microsoft.NETCore.App]
  Microsoft.NETCore.App 1.1.2 [/usr/local/share/dotnet/shared/Microsoft.NETCore.App]
  Microsoft.NETCore.App 2.0.0-preview2-25407-01 [/usr/local/share/dotnet/shared/Microsoft.NETCore.App]
  Microsoft.NETCore.App 2.0.0 [/usr/local/share/dotnet/shared/Microsoft.NETCore.App]
  Microsoft.NETCore.App 2.0.5 [/usr/local/share/dotnet/shared/Microsoft.NETCore.App]
  Microsoft.NETCore.App 2.1.1 [/usr/local/share/dotnet/shared/Microsoft.NETCore.App]
  Microsoft.NETCore.App 2.1.2 [/usr/local/share/dotnet/shared/Microsoft.NETCore.App]
  Microsoft.NETCore.App 2.1.7 [/usr/local/share/dotnet/shared/Microsoft.NETCore.App]
  Microsoft.NETCore.App 2.1.8 [/usr/local/share/dotnet/shared/Microsoft.NETCore.App]
  Microsoft.NETCore.App 2.1.9 [/usr/local/share/dotnet/shared/Microsoft.NETCore.App]
  Microsoft.NETCore.App 2.1.13 [/usr/local/share/dotnet/shared/Microsoft.NETCore.App]
  Microsoft.NETCore.App 2.2.3 [/usr/local/share/dotnet/shared/Microsoft.NETCore.App]
  Microsoft.NETCore.App 2.2.4 [/usr/local/share/dotnet/shared/Microsoft.NETCore.App]
  Microsoft.NETCore.App 3.0.0-preview8-28405-07 [/usr/local/share/dotnet/shared/Microsoft.NETCore.App]
  Microsoft.NETCore.App 3.0.0-rc1-19456-20 [/usr/local/share/dotnet/shared/Microsoft.NETCore.App]
  Microsoft.NETCore.App 3.0.0 [/usr/local/share/dotnet/shared/Microsoft.NETCore.App]
  Microsoft.NETCore.App 3.1.0-preview1.19506.1 [/usr/local/share/dotnet/shared/Microsoft.NETCore.App]

Windows info

$ dotnet --info
.NET Core SDK (reflecting any global.json):
 Version:   3.0.100
 Commit:    04339c3a26

Runtime Environment:
 OS Name:     Windows
 OS Version:  10.0.18362
 OS Platform: Windows
 RID:         win10-x64
 Base Path:   C:\Program Files\dotnet\sdk\3.0.100\

Host (useful for support):
  Version: 3.0.0
  Commit:  7d57652f33

.NET Core SDKs installed:
  1.1.11 [C:\Program Files\dotnet\sdk]
  2.1.202 [C:\Program Files\dotnet\sdk]
  2.1.403 [C:\Program Files\dotnet\sdk]
  2.1.500 [C:\Program Files\dotnet\sdk]
  3.0.100 [C:\Program Files\dotnet\sdk]

.NET Core runtimes installed:
  Microsoft.AspNetCore.All 2.1.5 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.All]
  Microsoft.AspNetCore.All 2.1.6 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.All]
  Microsoft.AspNetCore.App 2.1.5 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 2.1.6 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 3.0.0 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.NETCore.App 1.0.13 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 1.1.10 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 2.0.9 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 2.1.5 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 2.1.6 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 2.2.7 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 3.0.0 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.WindowsDesktop.App 3.0.0 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]

[atlemann]

I'm experiencing the same thing on Ubuntu.