Integrate LLD for ELF and Mach-O targets too

[kinke]

Requires/includes #2148 and additionally Requires LLD 5.0+. Still completely untested. ;)

[kinke]

Sadly doesn't work just yet on Linux:

% ldc2 -link-internally -v -run hello.d
...
ld.lld hello-547e688.o -o /tmp/hello-547e688-f0fc5d -L/home/martin/build-ldc/lib -lphobos2-ldc -ldruntime-ldc --gc-sections -lrt -ldl -lpthread -lm 
ldc2: /home/martin/llvm/lib/Support/CommandLine.cpp:1035: bool {anonymous}::CommandLineParser::ParseCommandLineOptions(int, const char* const*, llvm::StringRef, llvm::raw_ostream*): Assertion `hasOptions() && "No options specified!"' failed.

LLVM's command-line parser is first used by LDC, then later (in the same process) by LLD. Probably needs to be reset inbetween or something like that. I hoped the shared cmdline options would make LLD automatically inherit them (without having to add them explicitly to the LLD command line), so ideally those wouldn't be reset.

[kinke]

Same problem for MSVC targets; so LLVM 5.0 broke -link-internally for LDC.

[kinke]

LLD needs a patch; working locally.

[kinke]

Alright, I just tested linking a Phobos-hello-world with the [patched] integrated LLD 5.0 on Ubuntu 16.04. After adding a whole lotta linker flags normally added by gcc (and obtainable via gcc -v), it works fine and produces a somewhat smaller binary (876KB vs. 908KB) than system ld.bfd.
Exemplary command line:

ldc2 ../hello.d -link-internally -L--as-needed -L-dynamic-linker -L/lib64/ld-linux-x86-64.so.2 -L-z -Lrelro /usr/lib/gcc/x86_64-linux-gnu/5/../../../x86_64-linux-gnu/crt1.o /usr/lib/gcc/x86_64-linux-gnu/5/../../../x86_64-linux-gnu/crti.o /usr/lib/gcc/x86_64-linux-gnu/5/crtbegin.o -L-L/usr/lib/gcc/x86_64-linux-gnu/5 -L-L/usr/lib/gcc/x86_64-linux-gnu/5/../../../x86_64-linux-gnu -L-L/usr/lib/gcc/x86_64-linux-gnu/5/../../../../lib -L-L/lib/x86_64-linux-gnu -L-L/lib/../lib -L-L/usr/lib/x86_64-linux-gnu -L-L/usr/lib/../lib -L-L/usr/lib/gcc/x86_64-linux-gnu/5/../../.. -L-lm -L-lgcc -L--as-needed -L-lgcc_s -L--no-as-needed -L-lc -L-lgcc -L--as-needed -L-lgcc_s -L--no-as-needed /usr/lib/gcc/x86_64-linux-gnu/5/crtend.o /usr/lib/gcc/x86_64-linux-gnu/5/../../../x86_64-linux-gnu/crtn.o

LLD 5.0 patch, skipping parsing the (unused) -mllvm* options if none have been specified:

--- a/COFF/Driver.cpp
+++ b/COFF/Driver.cpp
@@ -696,7 +696,8 @@ void LinkerDriver::link(ArrayRef<const char *> ArgsArr) {
   V.push_back("lld-link (LLVM option parsing)");
   for (auto *Arg : Args.filtered(OPT_mllvm))
     V.push_back(Arg->getValue());
-  cl::ParseCommandLineOptions(V.size(), V.data());
+  if (V.size() > 1)
+    cl::ParseCommandLineOptions(V.size(), V.data());

   // Handle /errorlimit early, because error() depends on it.
   if (auto *Arg = Args.getLastArg(OPT_errorlimit)) {
--- a/ELF/Driver.cpp
+++ b/ELF/Driver.cpp
@@ -245,7 +245,8 @@ static void initLLVM(opt::InputArgList &Args) {
   V.push_back("lld (LLVM option parsing)");
   for (auto *Arg : Args.filtered(OPT_mllvm))
     V.push_back(Arg->getValue());
-  cl::ParseCommandLineOptions(V.size(), V.data());
+  if (V.size() > 1)
+    cl::ParseCommandLineOptions(V.size(), V.data());
 }

 // Some command line options or some combinations of them are not allowed.

---

LLVM's lazy-created globals scheme via ManagedStatic doesn't seem to work out for LDC and LLD (separate globals, at least different global command line parsers). We don't need the command-line parser as we don't specify -mllvm* options by default. LTO doesn't work for another reason (tested for both MSVC and Linux targets); it complains about uninitialized LLVM passes being registered (although we register that specific one earlier in LDC). There's a comment in LLVM src wrt. the global PassRegistry (IIRC) explicitly mentioning that it cannot be reset (e.g., via llvm::shutdown()). I tried a couple of things, nothing worked.

LTO on Windows works for a Phobos-hello-world when using an external LLD (lld-link.exe) [only the module itself, I haven't checked building and using LTO-enabled runtime libs yet]. Currently there's no command-line option to use an external linker other than link.exe for MSVC targets; we'll need to add one (-mslink=lld-link?). So I had to link manually.

[kinke]

Follow-up:

• LLD patch incorporated in LDC-LLVM 5.0.0-2 (incl. new LLD fork). AppVeyor, ldc-scripts and Docker image updated accordingly. => -link-internally working for MSVC targets with LDC master and LLD 5.0 again (no LTO). This PR enables it for Linux and OSX targets too, increasing the LDC executable size and being cumbersome to use due to the many additional linker command-line args. Still interesting for cross-linking.
• -mslink -linker implemented in Add -linker command-line option #2386. Compiling druntime/Phobos with -flto=full fails on Windows with LLVM 5.0.

[kinke]

Reviving this (with LLD 6.0.0 requirement, to simplify the CMake setup), as there's more interest recently (for cross-linking and possibly Android). The actually required linker flags aren't that many as in my example above (i.e., all default ones used by gcc), e.g., the following list of additional LDC flags should suffice to cross-link to Linux/ARM (based on https://forum.dlang.org/post/hlzedmugsxcrcoxpxbuj@forum.dlang.org):

-mtriple=arm-linux-gnueabihf,
-link-internally,
-L-m, -Larmelf_linux_eabi,
-L-sysroot, -L$ROOT,
-L$ROOT/usr/lib/arm-linux-gnueabihf/crt1.o,
-L$ROOT/usr/lib/arm-linux-gnueabihf/crti.o,
-L$ROOT/usr/lib/arm-linux-gnueabihf/crtbegin.o,
-L-L$ROOT/usr/lib/arm-linux-gnueabihf,
-L-L$ROOT/usr/lib/gcc/arm-linux-gnueabihf/4.9.2,
-L$ROOT/usr/lib/arm-linux-gnueabihf/crtend.o,
-L$ROOT/usr/lib/arm-linux-gnueabihf/crtn.o,
-L-lc,
-L-lgcc,
-L-lgcc_eh

[kinke]

On Linux x64 Ubuntu 16.04, with LLVM 6.0.0 and Release CMake config, LDC_WITH_LLD=ON increases the ldc2 executable size by 13% (+9 MB) over LDC_WITH_LLD=OFF. The existing MSCOFF-exclusive output capability cost ~7% back then, so the few additional MB on top for ELF and Mach-O seem well worth the effort.

Btw, LLD 6.0.0 ships with new support for MinGW and WebAssembly.