macOS release binaries not able to create executables

[myobie]

Given these env variables:

export PATH="${PROJECT}/wasi-sdk-10.0/bin:$PATH"
export WASI_SYSROOT="${PROJECT}/wasi-sdk-10.0/share/wasi-sysroot"
clang --target=wasm32-wasi --sysroot="${WASI_SYSROOT}" ...

I then see configure: error: C compiler cannot create executables.

• ./configure --prefix=/.../wasm-build --with-sysroot=/.../wasi-sdk-10.0/share/wasi-sysroot --host=wasm32-wasi --disable-ssp --disable-shared
  checking build system type... i386-apple-darwin19.5.0
  checking host system type... wasm32-unknown-wasi
  checking for a BSD-compatible install... /usr/bin/install -c
  checking whether build environment is sane... yes
  checking for wasm32-wasi-strip... llvm-strip
  checking for a thread-safe mkdir -p... build-aux/install-sh -c -d
  checking for gawk... gawk
  checking whether make sets $(MAKE)... yes
  checking whether make supports nested variables... yes
  checking whether UID '501' is supported by ustar format... yes
  checking whether GID '20' is supported by ustar format... yes
  checking how to create a ustar tar archive... gnutar
  checking whether make supports nested variables... (cached) yes
  checking whether to enable maintainer-specific portions of Makefiles... no
  checking whether make supports the include directive... yes (GNU style)
  checking for wasm32-wasi-gcc... clang
  checking whether the C compiler works... no
  configure: error: in /.../': configure: error: C compiler cannot create executables See config.log' for more details

I am able to make a sysroot myself inside a checked out wasi-sdk repo with make install INSTALL_DIR="${PROJECT}/sysroot/" and I can use it to compile the same project (after I've downloaded and placed libclang_rt.builtins-wasm32-wasi-10.0.tar.gz into my llvm installation inside homebrew). However, I'd like to take advantage of the pre-compiled binaries + sysroots if at all possible because I've run into a problem too many times with one machine having a different version of llvm installed, etc.

One key difference I notice when using the version I compiled myself is it says it's build system type is x86_64-apple-darwin19.5.0.

I am using macOS 10.15.5 with the latest homebrew and llvm 10.0.0_3.

[jedisct1]

The config.log file may tell you more about why executables cannot be created.

But macOS binaries available for download here are not notarized, so they cannot be used on Catalina if you download them using a web browser, or anything quarantining untrusted downloads.

A possible workaround is to download them using curl instead of a web browser.

But if you are using Homebrew, you really don't need a second installation of LLVM, especially since LLVM packages in Homebrew are always up to date.

The wasi-sdk downloads include a precompiled wasi-sysroot. This, in addition to the precompiled builtins, is all you need if you already installed llvm using homebrew.

I started to work on a Homebrew package to add these dependencies. A package for the wasi sysroot is pretty straightforward, but for the builtins, I couldn't figure out how to cleanly install them into another package's (llvm) path.

[pchickey]

Thanks @jedisct1 - we'd love to make notarized packages available for download but none of the current maintainers have the expertise (and most of us don't even have macs) to get it done, so we'd welcome help with that. Homebrew is another possibility, but similarly I don't have any idea how that actually works.

[jedisct1]

Another option is having a curl | sh installation script similar to rustup.

That wouldn't require notarization, would provide a unified installation procedure on all operating systems, and would provide a single-step way to install wasi-sdk, with or without LLVM, and with optional components (Lucet, Wasmtime, etc).

[sbc100]

We already provide a tarball containing just the sysroot if you want to use the llvm from homebrew: https://github.com/WebAssembly/wasi-sdk/releases/download/wasi-sdk-10/wasi-sysroot-10.0.tar.gz

[myobie]

I downloaded the release with wget and the binaries that were un-tarred are able to be executed, they however do not produce executables on my machine. How can I help better diagnose this issue?

[jedisct1]

clang --target=wasm32-wasi --sysroot="${WASI_SYSROOT}" ...

Is that version of clang before the the Homebrew (or Xcode) one in your $PATH?

If not, provide the full path to it, or set a temporary $PATH with the wasi-sdk version first.

[pchickey]

they however do not produce executables on my machine.

The default target is wasm32-wasi, so we would expect the file format of the executables to be a Wasm binary, with imports from the Wasi interface.

[sbc100]

Oh wait are you expecting to be able to run the output of the toolchain directly? Just to be clear this toolchain builds wasm files, which are not directly runnable. You need to run them in a wasm engine.

Assuming you already knew that can you build a simple hello world program, and include the full command line (with -v added) and the full output? Also can you attach the output file?

[jedisct1]

configure: error: C compiler cannot create executables doesn't mean that autoconf tried to execute the program, just that it wasn't able to build it.

[sbc100]

Right, I would imaging that is what that means.

I was just confused by the statement: "they however do not produce executables on my machine". What does the toolchain produce then? What happens when you run clang directly?

[myobie]

I apologize, I was just repeating the error statement.

[jedisct1]

The actual error is visible in the config.log file.

[sbc100]

@myobie when you use wget and you run clang directly from the command line (for example to build hello world) what error do you see?

[myobie]

@sbc100 I decided to create a simple example project today to isolate the issue and the included binaries do work on my Mac, so my and other's original suspicions were wrong. See https://gist.github.com/myobie/e5f15231cca4fc22e0b77a741ea64772

The project I am attempting to compile has a configure script and I dug into it today and it attempts to compile an example program and then looks for various possible output files like a.out, conftest.exe, etc. There are no wasm file names in its tests and since these release binaries only output wasm results, the configure script ends with the original error I posted above.

I manually edited the configure script to add conftest.wasm and a.wasm to the list of possible output file names and that allowed the script to complete. So I apologize, this appears to be an upstream error / assumption; however, it is very understandable how this could happen and it could happen to many projects that make similar tests / assumptions. I'm not sure how to fix this other than making a PR to the upstream project to change their scripts (and having a .patch for myself for now).

Thanks for your help. I'm going to close this, since the original issue wasn't really valid. However, I'll stay subscribed and if you all need or want any more info from me then let me know.