Fix shared libs for multilib macOS builds & tackle -rpath #2659

kinke · 2018-04-19T20:50:45Z

No description provided.

kinke · 2018-04-19T21:48:14Z

Oookay, so the macOS shared libs issue is even worse with master here, segfaulting during the d_do_test unittests, with all 32/64-bit debug/release libs:

1564: /Users/distiller/ninja-ldc/bin/ldmd2 -conf= -m64 -fPIC -g -unittest -run d_do_test.d -unittest
1564: 0   libdruntime-ldc-shared.dylib        0x000000010aa9943e _D4core7runtime18runModuleUnitTestsUZ19unittestSegvHandlerUNbiPSQCk3sys5posix6signal9siginfo_tPvZv + 62
1564: 1   libsystem_platform.dylib            0x00007fffa4042b3a _sigtramp + 26
1564: 2   libdruntime-ldc-shared.78.dylib     0x000000010abb74cf gc_query + 15
1564: 3   libdruntime-ldc-shared.78.dylib     0x000000010aba5e09 _D4core6memory2GC5queryFNaNbPvZSQBeQBc8BlkInfo_ + 9
1564: 4   libdruntime-ldc-shared.78.dylib     0x000000010abc54e8 _d_arraysetcapacity + 280
1564: 5   libphobos2-ldc-shared.dylib         0x000000010a63ca33 _D3std5regex8internal6parser7CodeGen5startMFkZv + 19
1564: 6   d_do_test-4ba7197-a198e4            0x000000010a38e2b2 _D3std5regex8internal6parser__T6ParserTAyaTSQBqQBpQBmQBg7CodeGenZQBi__T6__ctorTAxaZQmMFNcNeQCaQpZSQDsQDrQDoQDi__TQDeTQDaTQDaZQDq + 114
1564: 7   d_do_test-4ba7197-a198e4            0x000000010a38d89c _D3std5regex__T9regexImplTAyaZQpFNfQjAxaZSQBoQBn8internal2ir__T5RegexTaZQj + 124
1564: 8   d_do_test-4ba7197-a198e4            0x000000010a38da81 _D3std10functional__T7memoizeS_DQBe5regex__T9regexImplTAyaZQpFNfQjAxaZSQCrQBn8internal2ir__T5RegexTaZQjVii8ZQDjFQCfQBxZQBx + 433
1564: 9   d_do_test-4ba7197-a198e4            0x000000010a38d809 _D3std5regex__TQjTAyaZQqFNeAQkAxaZSQBhQBg8internal2ir__T5RegexTaZQj + 633
1564: 10  d_do_test-4ba7197-a198e4            0x000000010a36ce29 _D3std5regex__TQjTAyaZQqFNeQjAxaZSQBgQBf8internal2ir__T5RegexTaZQj + 105
1564: 11  d_do_test-4ba7197-a198e4            0x000000010a36a4da _D9d_do_test11unifyDirSepFAyaQdZQg + 138
1564: 12  d_do_test-4ba7197-a198e4            0x000000010a36ce69 _D9d_do_test26__unittest_d_do_test_387_0FZv + 41
1564: 13  d_do_test-4ba7197-a198e4            0x000000010a3e7e86 _D9d_do_test10__unittestZ + 6
1564: 14  libdruntime-ldc-shared.dylib        0x000000010aa994bb _D4core7runtime18runModuleUnitTestsUZ14__foreachbody2MFPS6object10ModuleInfoZi + 27
1564: 15  libdruntime-ldc-shared.dylib        0x000000010aabe1f2 _D2rt5minfo17moduleinfos_applyFMDFyPS6object10ModuleInfoZiZ14__foreachbody2MFKSQCz19sections_elf_shared3DSOZi + 50
1564: 16  libdruntime-ldc-shared.dylib        0x000000010aabf4aa _D2rt19sections_elf_shared3DSO7opApplyFMDFKSQBqQBqQyZiZi + 58
1564: 17  libdruntime-ldc-shared.dylib        0x000000010aabe19c _D2rt5minfo17moduleinfos_applyFMDFyPS6object10ModuleInfoZiZi + 28
1564: 18  libdruntime-ldc-shared.dylib        0x000000010aaaca4f _D6object10ModuleInfo7opApplyFMDFPSQBhQBdZiZi + 31
1564: 19  libdruntime-ldc-shared.dylib        0x000000010aa99274 runModuleUnitTests + 212
1564: 20  libdruntime-ldc-shared.dylib        0x000000010aab5b5f _D2rt6dmain211_d_run_mainUiPPaPUAAaZiZ6runAllMFZv + 31
1564: 21  libdruntime-ldc-shared.dylib        0x000000010aab5a61 _d_run_main + 433
1564: 22  d_do_test-4ba7197-a198e4            0x000000010a3e8015 main + 37

kinke · 2018-04-19T21:57:04Z

Oh wait, there's a mixture of libdruntime-ldc-shared.dylib (linked directly) and libdruntime-ldc-shared.78.dylib (probably referenced by libphobos2-ldc-shared.dylib), seemingly in different address spaces (i.e., loaded twice), and the segfault happens in the .78 one.

kinke · 2018-04-20T18:13:47Z

@JohanEngelen @klickverbot: install_name + symlink macOS fixes (for MULTILIB=ON) aside, I think that's a sane solution for rpath now. There'll be a default rpath (absolute path to the lib dir(s)) for produced binaries linked against the shared default libs now, for CMake-installed LDC too, so homebrewed builds and the official prebuilt packages (still portable) will produce shared libs and executables working out of the box. I guess it's fine the way it is for distros too, and they can simply adapt the .conf.in templates if need be.

I guess fiddling with rpath is now only really required when the user releases a (portable) binary linked against shared druntime/Phobos/ldc-jit, where the default rpath should be replaced by an appropriate relative one containing the shipped-with druntime/Phobos libs.

It's needed as otherwise the default rpath isn't set anymore with ldc-developers/ldc#2659.

kinke · 2018-04-20T20:42:21Z

CircleCI OSX now working fine with BUILD_SHARED_LIBS=ON: https://circleci.com/gh/ldc-developers/ldc/4789 (LTO integration test failures are expected due to implicit -link-defaultlib-shared and no -lto-shared libs) => CircleCI back to (default) BUILD_SHARED_LIBS=BOTH.

kinke · 2018-04-21T11:58:16Z

Switching from XCode 9.0 (9.0.1 supposedly) to 9.2 makes the CMake-installed libldc-jit-rt.a work according to the new integration test (edit: also verified manually via lipo -info).
Is that XCode upgrade okay for our prebuilt Mac packages?

Since using MACOSX_RPATH=ON for the 32/64-bit libs, the install_names of the individual libs already contain the `@rpath/` prefix (i.e., the built libs already, not just the CMake-installed copies). It is correctly retained when merging the libs via `lipo`. The actual issue was that the install_name contains the version too (`@rpath/libdruntime-ldc-shared.78.dylib`), so it needs to be retained in in order not to end up with duplicate druntime dependencies via prebuilt Phobos (depending on `@rpath/libdruntime-ldc-shared.78.dylib` from before the merging) and user-built binaries (linked against merged druntime with overridden install_name `@rpath/libdruntime-ldc-shared.dylib` without version number).

This mimics building an individual shared druntime/Phobos lib with CMake.

But still in a portable way for the portable prebuilt packages, which will replace the absolute install path with `%%ldcbinarypath%/..` in the config file, which will still lead to an absolute rpath in produced binaries.

This fixes the corrupt CMake-installed `libldc-jit-rt.a` issue ldc-developers#2618.

kinke force-pushed the test branch 4 times, most recently from 9305538 to ee10f07 Compare April 20, 2018 18:12

kinke changed the title ~~Test BUILD_SHARED_LIBS=ON for Circle Mac~~ Fix shared libs for multilib macOS builds & tackle -rpath Apr 20, 2018

kinke added a commit to ldc-developers/druntime that referenced this pull request Apr 20, 2018

LDC: Add -link-defaultlib-shared for appropriate standalone tests

28fea4f

It's needed as otherwise the default rpath isn't set anymore with ldc-developers/ldc#2659.

kinke force-pushed the test branch 2 times, most recently from 0670e3e to a48d15a Compare April 20, 2018 20:40

kinke mentioned this pull request Apr 20, 2018

OSX, dynamic compile: link errors with 1.8.0 package #2618

Closed

2 tasks

kinke force-pushed the test branch 4 times, most recently from a219099 to 397da88 Compare April 21, 2018 10:58

kinke mentioned this pull request Apr 21, 2018

Upgrade to D v2.079.1 #2587

Merged

kinke added 7 commits April 21, 2018 18:23

macOS: Generate & install symlinks for merged .dylibs

e894446

This mimics building an individual shared druntime/Phobos lib with CMake.

Link with default rpath set to absolute path to LDC lib dir(s)

611d324

But still in a portable way for the portable prebuilt packages, which will replace the absolute install path with `%%ldcbinarypath%/..` in the config file, which will still lead to an absolute rpath in produced binaries.

Only set rpath when linking against shared default libs

891234b

Set rpath with -enable-dynamic-compile too (shared ldc-jit lib)

71f3401

Refactor default libs linker stuff to driver/linker.cpp

82a6ccd

CircleCI Mac: Upgrade to XCode 9.2 and add dyncompile integration test

62d287e

This fixes the corrupt CMake-installed `libldc-jit-rt.a` issue ldc-developers#2618.

kinke force-pushed the test branch from 397da88 to 62d287e Compare April 21, 2018 16:24

kinke merged commit 964ad35 into ldc-developers:master Apr 22, 2018

kinke deleted the test branch April 22, 2018 12:26

kinke mentioned this pull request Apr 22, 2018

regression: OSX: Library not loaded: @rpath/libdruntime-ldc-shared.78.dylib #2615

Closed

kinke mentioned this pull request Apr 22, 2018

fix #2615 installed ldc.conf should set rpath #2616

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Fix shared libs for multilib macOS builds & tackle -rpath #2659

Fix shared libs for multilib macOS builds & tackle -rpath #2659

kinke commented Apr 19, 2018

kinke commented Apr 19, 2018 •

edited

Loading

kinke commented Apr 19, 2018

kinke commented Apr 20, 2018 •

edited

Loading

kinke commented Apr 20, 2018 •

edited

Loading

kinke commented Apr 21, 2018 •

edited

Loading

Fix shared libs for multilib macOS builds & tackle -rpath #2659

Fix shared libs for multilib macOS builds & tackle -rpath #2659

Conversation

kinke commented Apr 19, 2018

kinke commented Apr 19, 2018 • edited Loading

kinke commented Apr 19, 2018

kinke commented Apr 20, 2018 • edited Loading

kinke commented Apr 20, 2018 • edited Loading

kinke commented Apr 21, 2018 • edited Loading

kinke commented Apr 19, 2018 •

edited

Loading

kinke commented Apr 20, 2018 •

edited

Loading

kinke commented Apr 20, 2018 •

edited

Loading

kinke commented Apr 21, 2018 •

edited

Loading