Disable shared libraries when building with emscripten

[anutosh491]

Fixes #118048

[graphite-app]

Your org has enabled the Graphite merge queue for merging into main

Add the label “FP Bundles” to the PR and Graphite will automatically add it to the merge queue when it’s ready to merge.

You must have a Graphite account and log in to Graphite in order to use the merge queue. Sign up using this link.

[anutosh491]

Emscripten doesn't yet support dynamic linking as per what cmake expects so we can't build and use SHARED libraries in the normal way as mentioned by @sbc100 's comment here (emscripten-core/emscripten#17804 (comment))

Hence we could maybe have this instead to resolve the numerous warnings that show up while building against emscripten as shown in the issue.

Another way might be to have the following in all files that give warnings.

if(EMSCRIPTEN)
    set(LIB_TYPE STATIC)
else()
    set(LIB_TYPE SHARED or MODULE as expected)
endif()

add_llvm_library(XX ${LIB_TYPE} YY)

[anutosh491]

@sbc100 this might be ready

[anutosh491]

cc @MaskRay @sbc100

[sbc100]

I know there are other folks who are focused on building llvm for Wasm. I wonder why they haven't found a need to do this? Can you not perhaps globally configure llvm for static linking perhaps?

[anutosh491]

I mean I've seen people build it with warnings. The recipe hosted on emscripten-forge also has warnings but just ignores it. I just thought of addressing it here.

Can you not perhaps globally configure llvm for static linking perhaps?

I am guessing if there is a way to do this. It should be suggested maybe in the docs for llvm (does a section for building llvm for wasm already exist or do we need one in the first place ? .... if something like this exists in that case we might end up with such a suggestion. )

[sbc100]

@whitequark WDYT about this? IIUC you have been doing work making LLVM build for wasm.

[whitequark]

@sbc100 Seems reasonable, although I will note that I've focused on building LLVM for bare WASI since I need it to run on both JS-ful and JS-less platforms. As such I've never really used Emscripten much; pure-WASI takes different code paths here.

[anutosh491]

Thanks for the discussion here. I would just say that I have been playing around with building llvm for wasm through emscripten. And I see few errors when I do that, hence I raised this and #118051 (approved needs merge).

This would add more convinience rather than the long list of warnings that occur during the build. If we know we need to go for a static build, I guess we can possibly force it like what has been done here.

[anutosh491]

Also I had a question. Are there docs on how to build llvm (possibly clang, lld) for web assembly ?
I haven't come across one. If no, I would be happy to frame something for the same !

[whitequark]

I have a script building LLVM, Clang, and LLD here: https://github.com/YoWASP/clang

[anutosh491]

cc @sbc100

Do these changes look reasonable to you now ? I think @whitequark approves.
Obviously we might find work arounds here and there but shouldn't we cater specific configs needed by specific systems/platforms ?

I mean I see different setting based on checks like if(APPLE), If(MSVC) etc

But if that's not the case, whatever workaround is there should be well documented somewhere I guess. I've not had problems building llvm but then I see too warnings to ignore, hence in favour of such a change.

[sbc100]

@sbc100 Seems reasonable, although I will note that I've focused on building LLVM for bare WASI since I need it to run on both JS-ful and JS-less platforms. As such I've never really used Emscripten much; pure-WASI takes different code paths here.

And were you able to build without disabling shared libraries at the core level like this? (I assume you are not using any shared libraries in WASI since that is not really a thing yet).

[sbc100]

cc @sbc100

Do these changes look reasonable to you now ? I think @whitequark approves. Obviously we might find work arounds here and there but shouldn't we cater specific configs needed by specific systems/platforms ?

I mean I see different setting based on checks like if(APPLE), If(MSVC) etc

But if that's not the case, whatever workaround is there should be well documented somewhere I guess. I've not had problems building llvm but then I see too warnings to ignore, hence in favour of such a change.

If there are existing build settings that effectively disable shared libraries then we should probably just rely on them.

Are you able to make the build work without this patch using settings like these ones: https://github.com/YoWASP/clang/blob/0e0259397c85ba52b7370f6be0f52ae08ba3f4d5/build.sh#L72-L221

Can you update "web assembly" => "WebAssembly". Perhaps a better title might be "Disable shared libraries when building with emscripten"

[anutosh491]

cc @sbc100

I see the following variable responsible for building shared libs.

llvm-project/llvm/docs/CMake.rst

Line 314 in 1b9805c

**BUILD_SHARED_LIBS**:BOOL

I think that might be enough to get stuff working.
Would it make sense to enforce this in case of emscripten. Something like

if(EMSCRIPTEN)
    BUILD_SHARED_LIBS=OFF
endif()

EDIT: Oops sorry I thought this helped but actually this doesn't. I still see the warnings when building locally

CMake Warning (dev) at cmake/modules/AddLLVM.cmake:642 (add_library):
  ADD_LIBRARY called with SHARED option but the target platform does not
  support dynamic linking.  Building a STATIC library instead.  This may lead
  to problems.
Call Stack (most recent call first):
  /Users/anutosh491/work/llvm-project/clang/cmake/modules/AddClang.cmake:109 (llvm_add_library)
  /Users/anutosh491/work/llvm-project/clang/tools/clang-shlib/CMakeLists.txt:44 (add_clang_library)
This warning is for project developers.  Use -Wno-dev to suppress it.

..... around 10-15 more.....

CMake Warning (dev) at cmake/modules/AddLLVM.cmake:642 (add_library):
  ADD_LIBRARY called with SHARED option but the target platform does not
  support dynamic linking.  Building a STATIC library instead.  This may lead
  to problems.
Call Stack (most recent call first):
  cmake/modules/AddLLVM.cmake:942 (llvm_add_library)
  tools/remarks-shlib/CMakeLists.txt:16 (add_llvm_library)
This warning is for project developers.  Use -Wno-dev to suppress it.

[anutosh491]

Are you able to make the build work without this patch using settings like these ones: https://github.com/YoWASP/clang/blob/0e0259397c85ba52b7370f6be0f52ae08ba3f4d5/build.sh#L72-L221

Absolutely. The build is being hosted on emscripten-forge (which uses version 3.1.45 on main)

1. recipe.yaml (https://github.com/emscripten-forge/recipes/blob/main/recipes/recipes_emscripten/llvm/recipe.yaml)
2. build.sh (https://github.com/emscripten-forge/recipes/blob/main/recipes/recipes_emscripten/llvm/build.sh)

I realize why the above patch is required though. During llvm build process, if CMAKE_CROSSCOMPILING=ON, then a sub build of llvm
happens, a "native" build, to build the native tools needed to bootstrap the
compiler. Mostly llvm-tblgen and clang-tblgen, but they are not the only ones. So one way to get the build working is through providing your own version of the bootstrapped tool. I
did this by recompiling llvm locally to get llvm-tblgen and clang-tblgen, then
passing

    -DLLVM_TABLEGEN=$HOME/sources/llvm-project/_build/bin/llvm-tblgen \
    -DCLANG_TABLEGEN=$HOME/sources/llvm-project/_build/bin/clang-tblgen

But the approach I am using in emscripten-forge doesn't rely on the above.

[sbc100]

It seems perfectly reasonable to me to build tblgen for the host before compiling llvm for emscripten. Trying to pretend that emscripten is not a cross target is probably not worth the effort in this case.

[MaskRay]

BUILD_SHARED_LIBS defaults to OFF. ON builds shared objects like libLLVMFoo.so libLLVMBar.so. I don't think we should accept

if(EMSCRIPTEN)
    BUILD_SHARED_LIBS=OFF
endif()

You could report an error if EMSCRIPTEN is incomaptible with BUILD_SHARED_LIBS=on.

[llvm-beanz]

Seconding @MaskRay, if Emscripten doesn't support shared libraries we could error when you pass options like BUILD_SHARED_LIBS or LLVM_ENABLE_LLVM_DYLIB which cause LLVM to generate shared libraries, but we definitely should not be changing AddLLVM to magically shut off the library types.

You might want to make a change to CMake to detect the Emscrpiten target and error out if the Emscripten toolchain doesn't support shared libraries, but that should be done in CMake not LLVM's build system.