Add aarch64 build to releases

[LumiWasTaken]

Hiya,

we've been trying to deploy it on arm servers but ran into the issues of it looking for aarach64 versions... how much of a hassle would it be to add arm releases?

[kripken]

What type of aarch64 servers? We do build aarch64 for MacOS,

binaryen/.github/workflows/create_release.yml

Lines 53 to 55 in feb8f24

	- name: build-arm64
	run: cmake --build out-arm64 -v --config Release --target install
	if: matrix.os == 'macos-latest'

If you mean Linux then I'm not sure if GitHub Actions supports that, but maybe there has been an update I missed?

[LumiWasTaken]

Interesting, because i recall

- name: build-arm64 run: cmake --build out-arm64 -v --config Release --target install if: matrix.os == 'linux-latest'

Being a thing somewhere... but given that i have absolutely 0 idea about the actual workflows behind this maybe someone else knows more?

The issue is that it's trying to be smart and download
https://github.com/WebAssembly/binaryen/releases/download/version_116/binaryen-version_116-aarch64-linux.tar.gz

Which does not exist :/

[DazWorrall]

If you mean Linux then I'm not sure if GitHub Actions supports that, but maybe there has been an update I missed?

There are arm runners in private beta, but it looks like they will not be available to open source for the foreseeable (first going to Team and Enterprise plans).

I took a stab at building 116 on aarch64 using qemu via GitHub actions, but a) it's of course very slow, not sure if that would be acceptable and b) there was a test failure which needs attention from someone with more context than me!

[kripken]

I don't think speed would be an issue as this would only be for releases and not PRs, correct? Slower releases sound ok to me.

Btw, another option here might be the Zig cross-compilation toolchain? Emulation would be needed to run any tests, but that toolchain should be able to build aarch64 binaries at full speed at least.

The CI error looks like node: bad option: --experimental-wasm-threads which suggests it might be a too-old (or too-new?) node version. Finding which version is used there could help.

[DazWorrall]

I did also try building the tip of main a few weeks ago and got an actual build error fwiw, 116 built but failed the tests. Let me refresh my branch and try building main again, knowing that a slow release build would be acceptable means it's worth me spending a bit more time on this.

[DazWorrall]

Ok, tip of main still failing to build: https://github.com/DazWorrall/binaryen/actions/runs/7979012200/job/21785404126#step:9:240

/src/src/support/small_vector.h:32:38: error: '<unnamed>.wasm::SmallVector<wasm::Expression*, 10>::fixed' may be used uninitialized [-Werror=maybe-uninitialized]

Build is ok on x86_64 (or at least, the x86_64 version of the node:lts-alpine image). I can reproduce on my M2 Mac with podman.

[kripken]

Compiler error doesn't look arch-specific, that's odd. I would guess it is compiler-specific. That log has gcc 13.2.1 but I can't reproduce with 13.2.0 locally, so maybe it is arch-specific somehow..? Anyhow, if you find a reasonable workaround in the source code, we are ok with landing such fixes in general.

[DazWorrall]

The gcc version in both the aarch64 and x86_64 container is 13.2.1, and the x86_64 build works 🤷 Running ninja -v install, if I'm reading the logs right, there is no difference in the compiler flags:

# x86_64
[211/293]
/usr/bin/c++  -I/src/src -I/src/third_party/llvm-project/include -I/src -static -DBUILD_LLVM_DWARF -Wall -Werror -Wextra -Wno-unused-parameter -Wno-dangling-pointer -fno-omit-frame-pointer -fno-rtti -Wno-implicit-int-float-conversion -Wno-unknown-warning-option -Wswitch -Wimplicit-fallthrough -Wnon-virtual-dtor -fPIC -fdiagnostics-color=always -O3 -DNDEBUG -UNDEBUG -std=c++17 -MD -MT src/passes/CMakeFiles/passes.dir/Precompute.cpp.o -MF src/passes/CMakeFiles/passes.dir/Precompute.cpp.o.d -o src/passes/CMakeFiles/passes.dir/Precompute.cpp.o -c /src/src/passes/Precompute.cpp

# aarch64
ninja: job failed:
/usr/bin/c++  -I/src/src -I/src/third_party/llvm-project/include -I/src -static -DBUILD_LLVM_DWARF -Wall -Werror -Wextra -Wno-unused-parameter -Wno-dangling-pointer -fno-omit-frame-pointer -fno-rtti -Wno-implicit-int-float-conversion -Wno-unknown-warning-option -Wswitch -Wimplicit-fallthrough -Wnon-virtual-dtor -fPIC -fdiagnostics-color=always -O3 -DNDEBUG -UNDEBUG -std=c++17 -MD -MT src/passes/CMakeFiles/passes.dir/Precompute.cpp.o -MF src/passes/CMakeFiles/passes.dir/Precompute.cpp.o.d -o src/passes/CMakeFiles/passes.dir/Precompute.cpp.o -c /src/src/passes/Precompute.cpp

I am outside my wheelhouse here though. Maybe I've read the logs wrong.

The build works on 116, so something changed in here to upset the aarch64 builds: version_116...c0cdd26

I can help with the plumbing here but am not familiar at all with the source to suggest a fix.

[kripken]

If this used to work then bisecting to find the bad commit might help.

[DazWorrall]

Ok, I grabbed some native hardware for this (a ~90 minute emulated build time would have taken quite a while to bisect!): 141f7ca (#6212) is the first commit which fails to build on aarch64:

ninja: job failed: /usr/bin/c++  -I/root/binaryen/src -I/root/binaryen/third_party/llvm-project/include -I/root/binaryen -static -DBUILD_LLVM_DWARF -Wall -Werror -Wextra -Wno-unused-parameter -Wno-dangling-pointer -fno-omit-frame-pointer -fno-rtti -Wno-implicit-int-float-conversion -Wno-unknown-warning-option -Wswitch-Wimplicit-fallthrough -Wnon-virtual-dtor -fPIC -fdiagnostics-color=always -O3 -DNDEBUG -UNDEBUG -std=c++17 -MD -MT src/passes/CMakeFiles/passes.dir/Precompute.cpp.o -MF src/passes/CMakeFiles/passes.dir/Precompute.cpp.o.d -o src/passes/CMakeFiles/passes.dir/Precompute.cpp.o -c /root/binaryen/src/passes/Precompute.cpp
In file included from /root/binaryen/src/wasm-traversal.h:30,
                 from /root/binaryen/src/pass.h:24,
                 from /root/binaryen/src/ir/intrinsics.h:20,
                 from /root/binaryen/src/ir/effects.h:20,
                 from /root/binaryen/src/passes/Precompute.cpp:30:
In copy constructor 'wasm::SmallVector<wasm::Expression*, 10>::SmallVector(const wasm::SmallVector<wasm::Expression*, 10>&)',
    inlined from 'constexpr std::pair<_T1, _T2>::pair(const _T1&, const _T2&) [with _U1 = wasm::Select* const; _U2 = wasm::SmallVector<wasm::Expression*, 10>; typename std::enable_if<(std::_PCC<true, _T1, _T2>::_ConstructiblePair<_U1, _U2>() && std::_PCC<true, _T1, _T2>::_ImplicitlyConvertiblePair<_U1, _U2>()), bool>::type <anonymous> = true; _T1 = wasm::Select* const; _T2 = wasm::SmallVector<wasm::Expression*, 10>]' at /usr/include/c++/13.2.1/bits/stl_pair.h:559:21,
    inlined from 'T& wasm::InsertOrderedMap<Key, T>::operator[](const Key&) [with Key = wasm::Select*; T = wasm::SmallVector<wasm::Expression*, 10>]' at /root/binaryen/src/support/insert_ordered.h:112:29,
    inlined from 'void wasm::Precompute::partiallyPrecompute(wasm::Function*)::StackFinder::visitSelect(wasm::Select*)' at /root/binaryen/src/passes/Precompute.cpp:472:24,
    inlined from 'static void wasm::Walker<SubType, VisitorType>::doVisitSelect(SubType*, wasm::Expression**) [with SubType = wasm::Precompute::partiallyPrecompute(wasm::Function*)::StackFinder; VisitorType = wasm::Visitor<wasm::Precompute::partiallyPrecompute(wasm::Function*)::StackFinder, void>]' at /root/binaryen/src/wasm-delegations.def:50:1:
/root/binaryen/src/support/small_vector.h:32:38: error: '<unnamed>.wasm::SmallVector<wasm::Expression*, 10>::fixed' may be used uninitialized [-Werror=maybe-uninitialized]
   32 | template<typename T, size_t N> class SmallVector {
      |                                      ^~~~~~~~~~~
In file included from /root/binaryen/src/passes/Precompute.cpp:38:
/root/binaryen/src/support/insert_ordered.h: In static member function 'static void wasm::Walker<SubType, VisitorType>::doVisitSelect(SubType*, wasm::Expression**) [with SubType = wasm::Precompute::partiallyPrecompute(wasm::Function*)::StackFinder; VisitorType = wasm::Visitor<wasm::Precompute::partiallyPrecompute(wasm::Function*)::StackFinder, void>]':
/root/binaryen/src/support/insert_ordered.h:112:29: note: '<anonymous>' declared here
  112 |     std::pair<const Key, T> kv = {k, {}};
      |                             ^~
At global scope:
cc1plus: note: unrecognized command-line option '-Wno-unknown-warning-option' may have been intended to silence earlier diagnostics
cc1plus: note: unrecognized command-line option '-Wno-implicit-int-float-conversion' may have been intended to silence earlier diagnostics
cc1plus: all warnings being treated as errors
141f7cad3aa516db3828e619b31fe681e46a151b is the first bad commit

[DazWorrall]

I should add that the build above was ran in the same docker container used in CI (node:lts-alpine) for consistency, but I was also able to replicate on debian bullseye (gcc 10.2.1), reducing the chances that the issue is environmental.

[kripken]

Thanks, interesting... I see nothing in that commit that is special. But somehow that warning appears on that SmallVector usage...

To me this looks like a spurious warning. But it's beyond my C++ knowledge to guess as to why it complains here specifically, or to know how to work around it, sorry. Maybe you just need to disable warnings in general for this build?

[DazWorrall]

If you're happy with that I will try to do so conservatively.

[DazWorrall]

Proposing #6330