Add LDC_TARGET_PRESET CMake variable for runtime, to make cross-compilation easier

[joakim-noah]

The idea here is to be able to simply specify a dash-separated OS and CPU architecture and our CMake config will try to fill in the rest for you. Probably best to stick this in another CMake file, where we can have common flags for all the cross-compilation platforms the ldc devs have tried out, ie Linux/PPC, Windows/AArch64, or whatever else. You can always override RT_CFLAGS and LD_FLAGS by setting them manually, this logic will simply supply defaults for when the user doesn't.

Android is a little weird because CMake doesn't automatically detect if you're natively compiling on Android, and I can't set CMAKE_SYSTEM_NAME=Android because it will try to invoke the cross-compilation NDK toolchain. Basically, Android native compilation is unsupported by CMake, to the point where it just assumes cross-compilation and complains that it can't find the NDK.

Here are the commands I use to configure CMake for Android/ARM with this PR: note that I have to specify TARGET_PLATFORM LDC_TARGET_PRESET for native compilation too, because CMake just thinks native Android/ARM is Linux/ARM. These are the two use cases this PR currently supports, either natively compiling in the Termux Android app (tested by building ldc and running the druntime tests), or cross-compiling from linux/x64 (tested by building ldc and the druntime tests).

Native compilation

cmake .. -DLLVM_CONFIG=../../llvm-5.0.0rc2.src/build/bin/llvm-config -DTARGET_PLATFORM=Android-arm

Cross-compilation from linux/x64

CC=~/ndk-r15c/toolchains/llvm/prebuilt/linux-x86_64/bin/clang DMD=~/bin/dmd2/bin/dmd NDK=~/ndk-r15c/ cmake .. -DLLVM_CONFIG=../../llvm/build/bin/llvm-config -DTARGET_PLATFORM=Android-arm

What I'm looking for right now with this incomplete PR is what other ldc devs think of this approach, before I spend more time fleshing it out and adding flags for other architectures like Android-aarch64. For example, it currently assumes you're cross-compiling from linux/x64 using that NDK, whereas we'll need to support Windows/Mac too. I don't know if it's worth setting CMAKE_CROSSCOMPILING=True, which we haven't used yet, especially because ldc isn't cross-compiled so it may confuse CMake.

[joakim-noah]

Updated this pull to keep TARGET_SYSTEM as a public, cached config variable, though TARGET_PLATFORM will override it and D_FLAGS for setting the target triple. Only RT_CFLAGS and LD_FLAGS, of those four cross-compilation variables, can be set manually when TARGET_PLATFORM is used.

@kinke, what do you think? TARGET_PLATFORM just provides a simpler cross-compilation interface for your recent work, along with several preset configs, though users can always drop back to the four variables if wanted. I plan to add this as a flag for ldc-build-runtime too, to present a simpler interface for supported cross-compilation targets there too.

[kinke]

I think it's great for convenience and may get more people to try it out if they aren't confronted with your existing huge command line. ;)

I don't know any details about Android/ARM, so I don't know whether assuming ARMv7a and $NDK/toolchains/arm-linux-androideabi-4.9 is generic enough to work in most cases. And I don't know if the Android NDK for non-Linux hosts are compatible enough, so that it works on most hosts.

For Windows/MSVC targets, I guess it won't be that easy; I haven't heard of a full C++ cross-toolchain yet, so while clang and lld are most likely required (and available) on non-Windows hosts, I doubt the MS libraries are easily available.

For Linux x86(_64) targets, clang is probably a good choice as common C compiler, but I guess linker and libraries may be incompatible across distros/versions, and the linker may not be available on other hosts such as Windows (unless lld can really be used to target common Linux distros). But then the linker is only required for the shared runtime libs and testrunners, so being at least able to build the static runtime libs may suffice in many cases. In the future, the C libraries could be fetched from the distro's package repos (templates for Ubuntu x.y, Fedora x.y, Arch x.y...).

As to OSX, I have absolutely no idea/experience at all.

[kinke]

Wrt. linker, I forgot that we use the CMake archiver for the static runtime libs. We should probably use LDC instead, as it handles (cross-)archiving internally via LLVM.

[JohanEngelen]

(+1 for not using the default CMake archiver for static runtime libs. Generating LTO runtime libs is currently painful because of it; you have to override CMAKE_AR and CMAKE_RANLIB because the default tools most likely do not understand object files with LLVM IR in it)

[kinke]

Thanks Johan => #2309 so that we don't forget.

[joakim-noah]

I don't know any details about Android/ARM, so I don't know whether assuming ARMv7a and $NDK/toolchains/arm-linux-androideabi-4.9 is generic enough to work in most cases. And I don't know if the Android NDK for non-Linux hosts are compatible enough, so that it works on most hosts.

The NDK build script compiles for all Android-supported architectures by default, including x86, ARMv7, and ARMv5, but I see that they're deprecating the ARMv5 build in the upcoming r16 release, currently in beta. As for the gcc-toolchain, it's the only one they provide, so it will have to work. If you mean the version number, they very occasionally bump it, no big deal to match that. As for the Mac/Windows NDKs, haven't tried them, but doubt they're too different.

For Windows/MSVC targets, I guess it won't be that easy; I haven't heard of a full C++ cross-toolchain yet, so while clang and lld are most likely required (and available) on non-Windows hosts, I doubt the MS libraries are easily available.

Why would you need a "full C++ cross-toolchain?" We're just building a handful of C/ASM files for the D stdlib, just like any other platform. As for the MS C libraries, yeah, up to them to deal with that. We can only do so much.

As for Mac/linux, yeah, we can help them easily generate the stdlib static libraries, but after that it's up to them to supply C libraries, though we can make their life easier with your lld pull #2203.

[kinke]

Why would you need a "full C++ cross-toolchain?"

[I was already thinking about the full cross-compilation package incl. target C libs for full cross-linking support. C++ because I was already thinking about being able to cross-compile LDC itself. ;)]

[joakim-noah]

btw, I cross-compiled the ldc compiler itself for Android/ARM with the NDK for the first time when I wrote up the Termux build script a couple months ago, termux/termux-packages#1078, as I was only compiling the native compiler on my Android/ARM tablet before that. I haven't had a problem, so cross-compiling ldc itself at least works for Android/ARM, that too the ddmd 1.3 and 1.4 beta versions which require building a bootstrap ldc cross-compiler first.

[kinke]

I haven't had a problem, so cross-compiling ldc itself at least works for Android/ARM

Almost certainly only because you've cross-compiled from Linux to Android-Linux. LDC's CMake script still needs to be prepared for general cross-compilability.

[joakim-noah]

Yeah, but is there much left? I'll try it on wine with the Android NDK for Windows and let you know how it goes.

[kinke]

There are at least multiple if(MSVC) checks, and I guess that isn't set when using clang on Linux via CXX=clang (i.e., when cross-compiling LDC to MSVC).

Another likely problem is that we're invoking the llvm-config executable, and the linked-to target LLVM one's can generally not be invoked on an arbitrary host.

[joakim-noah]

Renamed TARGET_PLATFORM to PRESET_TARGET and moved the flags into a separate CMake module. I'll modify ldc-build-runtime to use this, try out the NDK on wine, and add some AArch64 flags before submitting this weekend.

Update: Oh, I wonder if it'd be worth adding the cross-compilation target triple and its settings to ldc2.conf automatically too?

[joakim-noah]

Don't love this, but the alternative was to stick the new CMake module in ../cmake/Modules and add that path here, so this config could be invoked on its own by ldc-build-runtime, but I believe that would make the top-level config look outside the ldc source directory, so figured this was better.

[kinke]

There's also CMAKE_CURRENT_SOURCE_DIR which should be able to get rid of the top-level CMakeLists.txt patch.

[joakim-noah]

OK, will look into that.

[joakim-noah]

Man, getting this running under Windows is a pain, as linux/x64->Android/ARM cross-compilation could use tools like the system ar and ranlib just fine, and the linux NDK toolchain would pass all the C/C++ tests that CMake puts it through, not so on Windows. Finally configured CMake so the druntime tests build and pass, now buillding the phobos tests under Wine.

[kinke]

#2319 hopefully works for cross-compilation scenarios too (requiring an -mtriple in D_FLAGS, which should always be required anyway).

[joakim-noah]

Alright, finally seems to be working well with this latest commit, running ldc-build-runtime in Wine right now with this command, after downloading Win64 binaries for the ldc 1.4 beta, CMake, the Windows/x64 NDK, and Ninja, checking out this git branch, setting CC, and adding CMake and Ninja to my path:

.\ldc-1.4\bin\ldc-build-runtime --ninja --testrunners --ldcSrcDir=C:\Users\joakim\ldc\ PRESET_TARGET=Android-arm CMAKE_C_COMPILER_WORKS=True CMAKE_SYSTEM_NAME=Linux

The last two flags were necessary to get cross-compiing to work. If I don't change the CMAKE_SYSTEM_NAME, CMake thinks I'm targeting Windows and adds some implib stuff to the final linking step. I can't set it to Android because that means diving into CMake's built-in cross-compilation support for Android, all for the few C/ASM files we compile, so Linux is the compromise. CMAKE_C_COMPILER_WORKS is needed to get it to stop running native tests of the C compiler, which for some hare-brained reason it runs even though I'm clearly cross-compiling.

Ideally, the last two shouldn't be necessary, but that's the minimum I had to do to wrestle CMake down. I'll add those to ldc-build-runtime when targeting Android, as they can't be set in our CMake config, too late in CMake's build process by then.

[kinke]

Nice. #2319 would/should render setting ar and ranlib obsolete.

[joakim-noah]

The default is C and CXX, but since we don't use C++ in the runtime, took it out. Otherwise, CMake insists on a C++ compiler and then runs some tests that break.

[joakim-noah]

Updated this pull to add flags for AArch64, though that 64-bit Android target runtime won't actually build right now, just setting us up for 1.5 and allowing me to extract the arch-dependent code out. Trying out #2319, works on Android/ARM and cross-compiling from linux/x64, will try it from Wine next.

Just need to push these new flags into ldc-build-runtime and play with supporting ldc2.conf a bit, then this pull should be done.

[kinke]

Defaulting to an empty string makes more sense to me, as it isn't auto-detected later.

[joakim-noah]

Will do.

[kinke]

Move this to line 50, so that the TARGET_SYSTEM condition doesn't change?

[joakim-noah]

They're mutually exclusive, see the first check in the included module, shouldn't matter. There is a possibility that someone might manually set both TARGET_SYSTEM and PRESET_TARGET to inconsistent systems, but this order won't affect that.

Or do you mean that TARGET_SYSTEM will be set by the included module before the above block? If so, it won't change anything either.

[kinke]

do you mean that TARGET_SYSTEM will be set by the included module before the above block?

Exactly, it is currently overwritten after bogus autodetection if a supported preset is used. So evaluating PRESET_TARGET before TARGET_SYSTEM autodetection makes more logical sense to me, and the existing condition for TARGET_SYSTEM-autodetection doesn't need to change.

[joakim-noah]

It shouldn't be overwritten, because the included module will always set TARGET_SYSTEM to something other than AUTO if PRESET_TARGET is set. If PRESET_TARGET is AUTO, the included CMake module does nothing. I fail to see a situation where both the above TARGET_SYSTEM auto-detection block and the new presets CMake module will be run, so the order won't matter.

[kinke]

Ah sry got that mixed up, there's the additional condition in the autodetection block (which bothers me) which prevents the overwrite. What I mean is that it makes just more sense to evaluate PRESET_TARGET first, to fully decouple the TARGET_SYSTEM autodetection.

[joakim-noah]

Ah, I see, run the module first, then I don't need to check for PRESET_TARGET in the auto-detection block, because TARGET_SYSTEM will be set. Makes sense, I only did it this way because I initially figured auto-detection should commonly take precedence, but I will reorder and remove the check, as you say.

[kinke]

Yep. It would also fix a bug: if PRESET_TARGET is an unsupported preset, currently it doesn't do anything (a warning wouldn't hurt I guess). So skipping the autodetection for PRESET_TARGET != AUTO leaves TARGET_SYSTEM at AUTO.

[kinke]

I'd prefer having this in the --presetTarget help text.

[joakim-noah]

I thought about adding a --listPresets flag, but considering it's basically just one that works right now, figured this is enough. Do we have time to add more preset configurations before the release, such as linux/PPC or linux/armhf, or do you still want to get it out this weekend?

[kinke]

I'm going on vacation, so I'd rather release pretty soon. LDC 1.5 isn't far, so I don't mind postponing a few non-crucial things. Not sure about the weekend though, the _d_dso_register stuff is making me feel somewhat at unease.

[joakim-noah]

Added an example for the flag above.

[dnadlinger]

Other projects have been moving towards prefixing all their CMake variables, e.g. LLVM_TARGETS_TO_BUILD. Maybe it is a good idea for us to start doing that as well, as it makes understanding the cache contents/ccmake output much easier, and reduces the chance of collisions with modules used. (We would of course still need to accept the existing vars transparently, but for newly introduced ones, this is something to consider.)

[joakim-noah]

Made all the changes asked for, just going to see what can be done with ldc2.conf and will remove the WIP tag.

I think it will cross-compile for Android on macOS/x64 and Win32, but I haven't tried those because I don't have them, so don't know about any incompatibiity there. Try it out and let me know.

[joakim-noah]

Hmm, the config file isn't generated if only the runtime is built and can't always be modified, say if part of a system package. Push that for later, this PR is ready to merge.

[dnadlinger]

Sorry for all the nitpicks, but what do you think about reversing the order to --targetPreset/TARGET_PRESET? Imho that would be much more suggestive ("It's a color preset/effect preset/target preset."), because the switch chooses a target preset to use.

[joakim-noah]

Makes sense, that's actually how I wrote it in the help text for ldc-build-runtime, took only a minute to switch with the appropriate regex substitions in vim.

[dnadlinger]

Thanks!

[dnadlinger]

It's fine like that, but FYI, std.algorithm.startsWith would also work.

[kinke]

Is the -mcpu=cortex-a8 not required anymore?

[joakim-noah]

It is, but only for llvm 5.0. I figured that since we're shipping 1.4 with 4.0.1, I'll worry about that later.