ld64.lld and duplicate symbols

[alucryd]

Hi guys, I'm trying to cross compile a lib (vips) that has several rust dependencies. Sadly the state of rust static libs seems to be getting worse with time and they are all plagued with duplicate symbols. I can work around that using -Wl,--allow-multiple-definition when targeting linux and windows, unfortunately targeting macos is a no-go because ld64.lld does not support that flag. Is there an alternative way of dealing with duplicate symbols on the ld64.lld side? I don't think rust static libs are going to be fixed anytime soon so my hands are tied.

duplicate symbol '___rust_foreign_exception' in:
    /home/embybuilder/Buildbot/armv8_osx/staging/lib/librav1e.a(std-2ca50e2550a11ed4.std.64f42f12-cgu.0.rcgu.o)
    /home/embybuilder/Buildbot/armv8_osx/libimagequant-armv8_osx/staging/lib/libimagequant.a(std-f6811f31090e0239.std.add96b74-cgu.0.rcgu.o)
duplicate symbol '_rust_begin_unwind' in:
    /home/embybuilder/Buildbot/armv8_osx/staging/lib/librav1e.a(std-2ca50e2550a11ed4.std.64f42f12-cgu.0.rcgu.o)
    /home/embybuilder/Buildbot/armv8_osx/libimagequant-armv8_osx/staging/lib/libimagequant.a(std-f6811f31090e0239.std.add96b74-cgu.0.rcgu.o)
duplicate symbol '___rdl_alloc_zeroed' in:
    /home/embybuilder/Buildbot/armv8_osx/staging/lib/librav1e.a(std-2ca50e2550a11ed4.std.64f42f12-cgu.0.rcgu.o)
    /home/embybuilder/Buildbot/armv8_osx/libimagequant-armv8_osx/staging/lib/libimagequant.a(std-f6811f31090e0239.std.add96b74-cgu.0.rcgu.o)
duplicate symbol '___rdl_realloc' in:
    /home/embybuilder/Buildbot/armv8_osx/staging/lib/librav1e.a(std-2ca50e2550a11ed4.std.64f42f12-cgu.0.rcgu.o)
    /home/embybuilder/Buildbot/armv8_osx/libimagequant-armv8_osx/staging/lib/libimagequant.a(std-f6811f31090e0239.std.add96b74-cgu.0.rcgu.o)
duplicate symbol '___rdl_dealloc' in:
    /home/embybuilder/Buildbot/armv8_osx/staging/lib/librav1e.a(std-2ca50e2550a11ed4.std.64f42f12-cgu.0.rcgu.o)
    /home/embybuilder/Buildbot/armv8_osx/libimagequant-armv8_osx/staging/lib/libimagequant.a(std-f6811f31090e0239.std.add96b74-cgu.0.rcgu.o)
duplicate symbol '___rdl_alloc' in:
    /home/embybuilder/Buildbot/armv8_osx/staging/lib/librav1e.a(std-2ca50e2550a11ed4.std.64f42f12-cgu.0.rcgu.o)
    /home/embybuilder/Buildbot/armv8_osx/libimagequant-armv8_osx/staging/lib/libimagequant.a(std-f6811f31090e0239.std.add96b74-cgu.0.rcgu.o)
duplicate symbol '_rust_panic' in:
    /home/embybuilder/Buildbot/armv8_osx/staging/lib/librav1e.a(std-2ca50e2550a11ed4.std.64f42f12-cgu.0.rcgu.o)
    /home/embybuilder/Buildbot/armv8_osx/libimagequant-armv8_osx/staging/lib/libimagequant.a(std-f6811f31090e0239.std.add96b74-cgu.0.rcgu.o)
duplicate symbol '___rust_drop_panic' in:
    /home/embybuilder/Buildbot/armv8_osx/staging/lib/librav1e.a(std-2ca50e2550a11ed4.std.64f42f12-cgu.0.rcgu.o)
    /home/embybuilder/Buildbot/armv8_osx/libimagequant-armv8_osx/staging/lib/libimagequant.a(std-f6811f31090e0239.std.add96b74-cgu.0.rcgu.o)
duplicate symbol '___rdl_oom' in:
    /home/embybuilder/Buildbot/armv8_osx/staging/lib/librav1e.a(alloc-ad1eeb16b5006415.alloc.c81d41f3-cgu.0.rcgu.o)
    /home/embybuilder/Buildbot/armv8_osx/libimagequant-armv8_osx/staging/lib/libimagequant.a(alloc-80ef5ce8d4cd7e19.alloc.157cdf10-cgu.0.rcgu.o)
duplicate symbol '___rg_oom' in:
    /home/embybuilder/Buildbot/armv8_osx/staging/lib/librav1e.a(std-2ca50e2550a11ed4.std.64f42f12-cgu.0.rcgu.o)
    /home/embybuilder/Buildbot/armv8_osx/libimagequant-armv8_osx/staging/lib/libimagequant.a(alloc-80ef5ce8d4cd7e19.alloc.157cdf10-cgu.0.rcgu.o)
ld: 10 duplicate symbols for architecture arm64
clang-15: error: linker command failed with exit code 1 (use -v to see invocation)

[rkettelerij]

Big +1 here, the linux (and I think windows) linker both support the --allow-multiple-definition flag. The linker on macOS doesn't, this makes it impossible to compile certain (cgo in my case) projects on macOS.

[cerisier]

The rabbit hole that led me to this issue, only to find out it's a dead end 😭

[theoparis]

I encountered the same issue just now trying to build mainline u-boot with -Wl,--allow-multiple-definition. This is currently needed because of duplicate symbol errors when linking the u-boot host tools on macOS.