Set up SemaphoreCI, tweak Travis config, speed up dmd-testsuite

[kinke]

Increase test coverage by including 32-bit shared libs, which for instance led to ldc-developers/druntime@a3af613.

[kinke]

Downside being that the chances for a green Travis run without 50 min timeouts for the multilib jobs will decrease further. :(

[kinke]

With 64-bit dmd-testsuite not tested for multilib jobs, we should be able to afford testing static+shared runtime libs for multilib jobs => changed accordingly, resulting in a LLVM 3.8 multilib job with static libs only and a 3.7 multilib job with static+shared libs.

[dnadlinger]

Thanks for doing this.

So far, we've gotten away without any issues, but with less and less of the full matrix being tested, we should also make sure (successfully) running the test suite is part of the release process. I guess we'll just add it to ldc-scripts? @redstar @JohanEngelen

[JohanEngelen]

+1 for adding running the testsuite to ldc-scripts

[kinke]

Stepped down from multilib & static+shared libs to multilib & shared-only libs job again, additionally using LLVM 3.6 for it, in order to minimize the timeouts in the unfortunate case of a slow Travis host.

[redstar]

I am also voting for a new release script which runs the test suite. Travis coverage is still good but my release build environment shows sometimes different problems. With a simple script to run the test suite we(I) could get more confidence in the distribution files.

[dnadlinger]

Are we going to use Semaphore for this?

[kinke]

Yep that's clearly the intention. ;) It's really blazing fast. But the 32-bit std.file release unittests consistently fail (static and shared); I already tried gcc 4.9 with the same result, so switched back to 4.8 default for the used Ubuntu 14.04. They use a RAM disk for I/O, so I'm wondering whether it's some race condition in a test (timestamps...).

[kinke]

The Semaphore issue is apparently an abort when unwinding during the std.file:3513 unittest:

Program received signal SIGABRT, Aborted.
0xf7fdb440 in __kernel_vsyscall ()
#0  0xf7fdb440 in __kernel_vsyscall ()
#1  0xf7db8687 in raise () from /lib/i386-linux-gnu/libc.so.6
#2  0xf7dbbab3 in abort () from /lib/i386-linux-gnu/libc.so.6
#3  0xf7f4f4f8 in ?? () from /lib/i386-linux-gnu/libgcc_s.so.1
#4  0xf7f4f907 in _Unwind_Resume () from /lib/i386-linux-gnu/libgcc_s.so.1
#5  0x082873d3 in _d_eh_resume_unwind ()
#6  0x081b534c in std.file.__unittestL3513_56() ()

I cannot reproduce it with my Ubuntu 16.04.

[dnadlinger]

@kinke: We should probably also look into getting a Release+Asserts LLVM build running on Semaphore.

[kinke]

Oh wow SemaphoreCI just passed for the first time! Don't know whether that was just a fluke, the proper default inlining threshold just hiding it or whether Semaphore changed something.

With SemaphoreCI working, we'd have a speedy-Gonzalez multilib CI setup. It's partially parallelized; 2 jobs, both building LDC, then one job testing dmd-testsuite and the other job running the other tests (LDC unittests, lit-tests, library unittests), taking 2x20 mins => 20 mins wall-clock time. So the quickest feedback from any CI system, and that for the most challenging/comprehensive (multilib, static+shared libs) environment. Awesome if I may say so. :)

[kinke]

Reset to a Travis config adaptation without multilib job.

[kinke]

I upgraded SemaphoreCI to LLVM 4.0.1 (the config is specified on their web page, they sadly don't support a config file in the source tree), works. Should also work for Travis Linux, although the 4.0.1 job once again timed out before running the library unittests.
LDC doesn't link for Travis on OSX with LLVM 4.0.1 though, complaining about a missing _futimens function. I hope it's not a LLVM regression (reincarnation of LLVM issue 16442?), maybe it (annoyingly) 'just' requires a newer OSX now. OSX guys please check. :)

[kinke]

Just found a way to skip some redundant dmd-testsuite tests, reducing dmd-testsuite runtimes by 30% (for both debug and release) on my box, shaving off a total of 160 secs. Travis better like it.

[kinke]

Oh yeah baby, SemaphoreCI done in less than 14 mins:

The following tests passed:
	clean-dmd-testsuite-debug
	dmd-testsuite-debug
	clean-dmd-testsuite
	dmd-testsuite
	clean-dmd-testsuite-debug_32
	dmd-testsuite-debug_32
	clean-dmd-testsuite_32
	dmd-testsuite_32

100% tests passed, 0 tests failed out of 8

Total Test time (real) = 379.96 sec

The 5 previous master builds on Semaphore all took > 800 secs for these 8 tests!

[JohanEngelen]

Very nice! :-)
@kinke Could you try to add LTO testing on Semaphore? (did you add -DLDC_INSTALL_LTOPLUGIN=ON to cmake?)

[kinke]

I added -DLDC_INSTALL_LTOPLUGIN=ON and retriggered Semaphore, but it's apparently not finding ${LLVM_LIBRARY_DIRS}/LLVMgold.so...
Edit: Apparently called libLTO.so in the LLVM 3.9.1, 4.0.0 and 4.0.1 Linux (both Debian 8 and Ubuntu 14.04) archives, or is that something else and the LTO ld.gold plugin just not shipped in the Linux archives?

[JohanEngelen]

libLTO.so is something else. The plugin is not shipped in the Linux archives, probably because of binary issues with all kinds of Linuxes. On CircleCI, we use the APT repository, and that one does contain the plugin.
So unless we build LLVM ourselves with -DLLVM_BINUTILS_INCDIR=... or we use the APT repo, LTO is off the table.
Sorry for the noise.

[kinke]

=> #2189