Publishing release as single file does not include all libraries (libgssapi_krb5)

[kalaskarsanket]

---

Issue moved from microsoft/dotnet#1273

• Please respond to @lastcoolnameleft.

---

From @lastcoolnameleft on Monday, November 23, 2020 4:42:46 PM

I'm trying to create a single-file binary to run on Linux; however, it does not include all libraries. I'm unclear why the Kerberos library is needed.

To reproduce:

➜  dotnet --version
5.0.100
➜  dotnet new console -o single-file
The template "Console Application" was created successfully.

Processing post-creation actions...
Running 'dotnet restore' on single-file/single-file.csproj...
  Determining projects to restore...
  Restored /Users/thfalgou/tmp/dotnet/single-file/single-file.csproj (in 64 ms).
Restore succeeded.
➜  cd single-file 
➜  dotnet publish -r linux-x64 -p:PublishSingleFile=true -c Release
Microsoft (R) Build Engine version 16.8.0+126527ff1 for .NET
Copyright (C) Microsoft Corporation. All rights reserved.

  Determining projects to restore...
  Restored /Users/thfalgou/tmp/dotnet/single-file/single-file.csproj (in 89 ms).
  single-file -> /Users/thfalgou/tmp/dotnet/single-file/bin/Release/net5.0/linux-x64/single-file.dll
  single-file -> /Users/thfalgou/tmp/dotnet/single-file/bin/Release/net5.0/linux-x64/publish/                                                                           
➜  file bin/Release/net5.0/linux-x64/publish/single-file
bin/Release/net5.0/linux-x64/publish/single-file: ELF 64-bit LSB pie executable, x86-64, version 1 (GNU/Linux), dynamically linked, interpreter /lib64/ld-linux-x86-64.so.2, for GNU/Linux 2.6.32, BuildID[sha1]=6ca69f7bc7061d8eab20013d6820ef5a5c2f7ff4, stripped, too many notes (256)

Start an Ubuntu instance via docker

➜  docker run -it -v $PWD/bin/Release/net5.0/linux-x64/publish:/tmp/single-file ubuntu bash
root@e91793747e67:/# cd /tmp/single-file/
root@e91793747e67:/tmp/single-file# ls
single-file  single-file.pdb
root@e91793747e67:/tmp/single-file# ./single-file
./single-file: error while loading shared libraries: libgssapi_krb5.so.2: cannot open shared object file: No such file or directory
root@e91793747e67:/tmp/single-file# ldd ./single-file
	linux-vdso.so.1 (0x00007ffc371c4000)
	libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f1b744cd000)
	libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007f1b744c7000)
	libz.so.1 => /lib/x86_64-linux-gnu/libz.so.1 (0x00007f1b744ab000)
	libgssapi_krb5.so.2 => not found
	librt.so.1 => /lib/x86_64-linux-gnu/librt.so.1 (0x00007f1b744a0000)
	libstdc++.so.6 => /lib/x86_64-linux-gnu/libstdc++.so.6 (0x00007f1b742bf000)
	libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f1b7416e000)
	libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007f1b74153000)
	libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f1b73f61000)
	/lib64/ld-linux-x86-64.so.2 (0x00007f1b75126000)

As you can see, libgssapi_krb5.so.2 was not included.

Due to -p:PublishSingleFile=true, I would expect the application to be a statically linked library instead of a dynamically linked one.

[kalaskarsanket]

---

Issue moved from microsoft/dotnet#1273

• Please respond to @lastcoolnameleft.

---

From @lastcoolnameleft on Monday, November 23, 2020 5:40:27 PM

FYI, adding --self-contained doesn't fix the issue.

Tagging subscribers to this area: @agocke, @vitek-karas
See info in area-owners.md if you want to be subscribed.

Issue Details

---

---

Issue moved from microsoft/dotnet#1273

• Please respond to @lastcoolnameleft.

---

From @lastcoolnameleft on Monday, November 23, 2020 4:42:46 PM

I'm trying to create a single-file binary to run on Linux; however, it does not include all libraries. I'm unclear why the Kerberos library is needed.

To reproduce:

➜  dotnet --version
5.0.100
➜  dotnet new console -o single-file
The template "Console Application" was created successfully.

Processing post-creation actions...
Running 'dotnet restore' on single-file/single-file.csproj...
  Determining projects to restore...
  Restored /Users/thfalgou/tmp/dotnet/single-file/single-file.csproj (in 64 ms).
Restore succeeded.
➜  cd single-file 
➜  dotnet publish -r linux-x64 -p:PublishSingleFile=true -c Release
Microsoft (R) Build Engine version 16.8.0+126527ff1 for .NET
Copyright (C) Microsoft Corporation. All rights reserved.

  Determining projects to restore...
  Restored /Users/thfalgou/tmp/dotnet/single-file/single-file.csproj (in 89 ms).
  single-file -> /Users/thfalgou/tmp/dotnet/single-file/bin/Release/net5.0/linux-x64/single-file.dll
  single-file -> /Users/thfalgou/tmp/dotnet/single-file/bin/Release/net5.0/linux-x64/publish/                                                                           
➜  file bin/Release/net5.0/linux-x64/publish/single-file
bin/Release/net5.0/linux-x64/publish/single-file: ELF 64-bit LSB pie executable, x86-64, version 1 (GNU/Linux), dynamically linked, interpreter /lib64/ld-linux-x86-64.so.2, for GNU/Linux 2.6.32, BuildID[sha1]=6ca69f7bc7061d8eab20013d6820ef5a5c2f7ff4, stripped, too many notes (256)

Start an Ubuntu instance via docker

➜  docker run -it -v $PWD/bin/Release/net5.0/linux-x64/publish:/tmp/single-file ubuntu bash
root@e91793747e67:/# cd /tmp/single-file/
root@e91793747e67:/tmp/single-file# ls
single-file  single-file.pdb
root@e91793747e67:/tmp/single-file# ./single-file
./single-file: error while loading shared libraries: libgssapi_krb5.so.2: cannot open shared object file: No such file or directory
root@e91793747e67:/tmp/single-file# ldd ./single-file
	linux-vdso.so.1 (0x00007ffc371c4000)
	libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f1b744cd000)
	libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007f1b744c7000)
	libz.so.1 => /lib/x86_64-linux-gnu/libz.so.1 (0x00007f1b744ab000)
	libgssapi_krb5.so.2 => not found
	librt.so.1 => /lib/x86_64-linux-gnu/librt.so.1 (0x00007f1b744a0000)
	libstdc++.so.6 => /lib/x86_64-linux-gnu/libstdc++.so.6 (0x00007f1b742bf000)
	libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f1b7416e000)
	libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007f1b74153000)
	libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f1b73f61000)
	/lib64/ld-linux-x86-64.so.2 (0x00007f1b75126000)

As you can see, libgssapi_krb5.so.2 was not included.

Due to -p:PublishSingleFile=true, I would expect the application to be a statically linked library instead of a dynamically linked one.

Author:	kalaskarsanket
Assignees:	-
Labels:	area-Single-File, untriaged
Milestone:	-

[agocke]

Single-file doesn't change the machine dependencies required by dotnet, it only links the dotnet libraries themselves together. You'll still need to install everything in https://docs.microsoft.com/en-us/dotnet/core/install/linux-ubuntu#dependencies

[agocke]

@danmosemsft Do you happen to know if we have a doc on why certain dotnet dependencies are machine dependencies, vs distributed w/ dotnet?

[danmoseley]

I do not know of a document that describes all our dependencies. It may be these simply have to be uncovered by testing. Perhaps @wfurt has context on this one.

[vitek-karas]

The main difference between single-file and non-single-file is that the superhost currently requires all of its dependencies - so if they are not present, that app won't even start. Normal apphost doesn't have this, the dependencies are only loaded when the relevant managed code is executed - which may never happen for some apps. So while it's technically correct that the dependency set hasn't changed, the effective behavior does change for certain apps.

[wfurt]

do we know where this comes from? AFAIK, all the dependencies from NegotiateStream should be hidden behind Native shim we dlopen when needed. SO I would expect that libgssapi_krb5 is NOT needed to run.

[vitek-karas]

In single-file we can't bundle the native shims as .so files, because we don't want to write them to disk on execution and we can't 'dlopen' them from the single-file. So we statically link all the native shims into the executable. But that means they are not delay-loaded anymore... and that's where the problem comes from.

[wfurt]

libSystem.Net.Security.Native.so:
        linux-vdso.so.1 (0x00007ffc7ed01000)
        libgssapi_krb5.so.2 => /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2 (0x00007fa64efee000)
        libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007fa64ebfd000)
        libkrb5.so.3 => /usr/lib/x86_64-linux-gnu/libkrb5.so.3 (0x00007fa64e927000)
        libk5crypto.so.3 => /usr/lib/x86_64-linux-gnu/libk5crypto.so.3 (0x00007fa64e6f5000)
        libcom_err.so.2 => /lib/x86_64-linux-gnu/libcom_err.so.2 (0x00007fa64e4f1000)
        libkrb5support.so.0 => /usr/lib/x86_64-linux-gnu/libkrb5support.so.0 (0x00007fa64e2e6000)
        /lib64/ld-linux-x86-64.so.2 (0x00007fa64f43d000)
        libkeyutils.so.1 => /lib/x86_64-linux-gnu/libkeyutils.so.1 (0x00007fa64e0e2000)
        libresolv.so.2 => /lib/x86_64-linux-gnu/libresolv.so.2 (0x00007fa64dec7000)
        libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007fa64dca8000)
        libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007fa64daa4000)

note that if we link libSystem.Net.Security.Native.so directly, we will not only include it's dependencies but recursively also their dependencies.

cc: @janvorli for ideas how to do this programatically.