1.0.0-beta2: Fatal error in EH code: _Unwind_RaiseException

[jondegenhardt]

One of my programs is exiting with:

    Fatal error in EH code: _Unwind_RaiseException failed with reason code: 5
    Abort trap: 6

Haven't seen this before, including on beta-1. This is on OSX. I haven't had time to narrow down a small use case yet. Apologies for that, but wanted to get an initial report out first, will try to generate a small reproducible case later, but might be a day before I can get to it.

General situation: It appears to be triggered via statements like:

    throw new Exception(format("--format string--", value, value...)

from within an option handler (function) passed to std.getopt.getopt, with the getopt call within a try-catch expression. My programs have other occurrences of 'throw new Exception' within a getopt call, the ones failing are cases where a function was passed to getopt to handle the argument.

Build info:

$ ldc2 --version
LDC - the LLVM D compiler (1.0.0-beta2):
  based on DMD v2.070.2 and LLVM 3.8.0
  built with LDC - the LLVM D compiler (0.17.1)
  Default target: x86_64-apple-darwin14.5.0
  Host CPU: haswell
  http://dlang.org - http://wiki.dlang.org/LDC

The failure is occurring in a publicly available program. To try to reproduce, clone the repository at: https://github.com/eBay/tsv-utils-dlang. Then:

    $ cd tsv-utils-dlang/tsv-filter
    $ make DCOMPILER=ldc2 test    # Tests will fail
    $ cat tests/latest_debug/error_tests_1.txt

The error_tests_1.txt file lists the command used and the error. Look for the ones starting with "Fatal error in EH code". The same commands can be run by hand. eg.

    $ cd tsv-utils-dlang/tsv-filter
    $ make DCOMPILER=ldc2
    $ ../bin/tsv-filter --gt 0:10 tests/input1.tsv
    Fatal error in EH code: _Unwind_RaiseException failed with reason code: 5
    Abort trap: 6

[dnadlinger]

Which Xcode version are you using? This might be #1031

[jondegenhardt]

Xcode Version 7.2.1 (7C1002)

[jondegenhardt]

In #1031 it appears there might be some interaction with debug builds. I'm getting the same failure in both debug and release builds. The automated test I listed is hitting the debug version test, it runs before the release test. However, the manual steps listed are against a release build.

[dnadlinger]

The issue is just related to the number of entries emitted into a certain unwind info table. Debug builds just happen to sometimes produce more different debug frame types.

[dnadlinger]

The linker bug might be fixed in 7.3.1 (Apple didn't confirm any details, as usual), could you easily give that a try?

[jondegenhardt]

Should be able to. It'll be a few hours before I can get to it though. I'll report back when I've had an opportunity.

[dnadlinger]

I get (on ldc master, built against LLVM 3.8.1, Xcode 7.3.1):

Error processing command line arguments: Invalid option: '--gt 0:10'. Zero is not a valid field index.

[jondegenhardt]

Yup, that's the correct result

[dnadlinger]

(In case you are wondering, whether and how the bug is triggered depends heavily on the codegen. IIRC -beta2 is the first OS X binary built against 3.8, which might explain the issue. @redstar: I think this has been brought up somewhere already, but a note that Xcode 7.3.1 needs to be used to avoid some toolchain bugs (e.g. #1031) should go into the release announcement somehow.)

[jondegenhardt]

Thanks, I thought LLVM version change might be involved.
I had hoped to upgrade Xcode and try again before I had to take off, but AppStore is taking a while. I should be able to try it in a few hours though.

[jondegenhardt]

Looks like I cannot upgrade to Xcode 7.3.1, as the 7.3.1 release requires first upgrading to OSX 10.11 (El Capitan). I'm on OSX 10.10 (Yosemite), and cannot upgrade to 10.11 at this time.

[jondegenhardt]

Good news (likely). Until a few days ago I was setting environment variables CC and CXX. When I stopped setting these this issue showed up. It was accidental that I found it when trying beta-2. It also happens with beta-1 and alpha-1. I didn't try any earlier LDC releases.

So, the good part is that it's not newly introduced with beta-2. Further, I probably created a linker version issue at the same time. I need to check further on this.

I was setting the environment variables to the version of GNU GCC I use, which is not the apple supplied GCC. With the environment variables set (CC=gcc-mp-4.9, CXX=g++-mp-4.9), everything was fine. Unset, the failures occur. The environment variables are used in makefile, apple's gcc is still on the path. However, there are also two version of ld, and I had the GNU version first on the path, not the apple version.

I'll explore my setup further and see if I can resolve this fully in my environment.

[jondegenhardt]

After a bit of trying, I'm unable to identify a setup that works, aside from setting the CC/CXX environment variables as described. It perhaps makes sense to assume it's something somewhat specific to my environment, or that Xcode 7.3.1 is required for several releases now. With the idea that it's specific to my environment, I'd appreciate some help figuring out what is going on. That would also help ensure the problem is not more widespread.

One thing that would help is a description of how ldc2 finds and invokes the linker, and how the CC/CXX environment variables are coming into play. There are two versions of ld on my system, plus gcc. gcc is always in PATH, and I tried putting both versions of ld in the PATH, but all combinations resulting in the same failure.

Details:

$ /usr/bin/ld -v
@(#)PROGRAM:ld  PROJECT:ld64-253.9
configured to support archs: armv6 armv7 armv7s arm64 i386 x86_64 x86_64h armv6m armv7k armv7m armv7em (tvOS)
LTO support using: Apple LLVM 7.0.2 (clang-700.1.81)

$ /opt/local/bin/ld -v
@(#)PROGRAM:ld  PROJECT:ld64-253.9
configured to support archs: i386 x86_64 x86_64h armv6 armv7 armv7s armv7m armv7k arm64 (tvOS)
LTO support using: LLVM version 3.5.2

$ /usr/bin/gcc -v
Configured with: --prefix=/Applications/Xcode.app/Contents/Developer/usr --with-gxx-include-dir=/usr/include/c++/4.2.1
Apple LLVM version 7.0.2 (clang-700.1.81)
Target: x86_64-apple-darwin14.5.0
Thread model: posix

$ $CC -v
Using built-in specs.
COLLECT_GCC=gcc-mp-4.9
COLLECT_LTO_WRAPPER=/opt/local/libexec/gcc/x86_64-apple-darwin14/4.9.3/lto-wrapper
Target: x86_64-apple-darwin14
Configured with: /opt/local/var/macports/build/_opt_mports_dports_lang_gcc49/gcc49/work/gcc-4.9.3/configure --prefix=/opt/local --build=x86_64-apple-darwin14 --enable-languages=c,c++,objc,obj-c++,lto,fortran,java --libdir=/opt/local/lib/gcc49 --includedir=/opt/local/include/gcc49 --infodir=/opt/local/share/info --mandir=/opt/local/share/man --datarootdir=/opt/local/share/gcc-4.9 --with-local-prefix=/opt/local --with-system-zlib --disable-nls --program-suffix=-mp-4.9 --with-gxx-include-dir=/opt/local/include/gcc49/c++/ --with-gmp=/opt/local --with-mpfr=/opt/local --with-mpc=/opt/local --with-isl=/opt/local --disable-isl-version-check --with-cloog=/opt/local --disable-cloog-version-check --enable-stage1-checking --disable-multilib --enable-lto --enable-libstdcxx-time --with-as=/opt/local/bin/as --with-ld=/opt/local/bin/ld --with-ar=/opt/local/bin/ar --with-bugurl=https://trac.macports.org/newticket --with-pkgversion='MacPorts gcc49 4.9.3_0'
Thread model: posix
gcc version 4.9.3 (MacPorts gcc49 4.9.3_0)

[dnadlinger]

It perhaps makes sense to assume […] that Xcode 7.3.1 is required for several releases now

This is more or less the case, although it's less that we suddenly started requiring Xcode 7.3.1, but that a number of prior ld64 versions contain said bug. You unfortunately can't expect LDC (or any other compiler that produces a lot of different unwind table entries) to work reliably with any prior version, although you might get "lucky" with some of the older LDC versions or ones built differently in that they might not end up triggering the bug in a visible way.

In short, I'm not sure what we can do about this, except for making sure that we prominently tell users about this. [1] If you are curious, here is my original patch against ld64-242, which I also included with my original Apple bug report: https://github.com/klickverbot/ld64/tree/compact-uwtable-fix. In theory, you should be able to build ld64 from source yourself with the patch applied (the 7.3 source archive has not been uploaded so far).

One thing that would help is a description of how ldc2 finds and invokes the linker

Check out ldc2 -v <…> – it just invokes gcc as searched for in the hidden -gcc command line option, the CC environment variable, and finally the PATH (the latter using llvm::sys::findProgramByName).

[1] To make sure users don't run into this, we could also add a step to the compiler that reads back the ld -v result and emits a warning if the version is too small. Not sure whether this is worth the overhead, though.

[jondegenhardt]

Thanks, very helpful. When I build with ldc2 -v, the failure case is building with the apple supplied gcc (Xcode). Building with gcc-mp-4.9 works fine.

So yes, perhaps issue the release with a requirement of Xcode 3.7.1 & OSX 10.11 (El Capitan). If gcc 4.9 is an option for Yosemite users, that path could be described as well.