conf-openblas does not build on macos ARM due to unsupported '-fopenmp'

[nilsbecker]

this is the error i get:

#=== ERROR while compiling conf-openblas.0.2.2 ================================#
# context     2.3.0 | macos/arm64 | ocaml-option-flambda.1 ocaml-option-no-flat-float-array.1 ocaml-variants.5.3.0+options | https://opam.ocaml.org#717e57b54bb7da8f42974d56f9408e807e4a154f
# path        ~/.opam/53/.opam-switch/build/conf-openblas.0.2.2
# command     ~/.opam/opam-init/hooks/sandbox.sh build sh -exc cc $CFLAGS $(PKG_CONFIG_PATH="$(brew --prefix openblas)/lib/pkgconfig:$PKG_CONFIG_PATH" pkg-config --cflags openblas) test.c $(PKG_CONFIG_PATH="$(brew --prefix openblas)/lib/pkgconfig:$PKG_CONFIG_PATH" pkg-config --libs openblas)
# exit-code   1
# env-file    ~/.opam/log/conf-openblas-84329-72df6a.env
# output-file ~/.opam/log/conf-openblas-84329-72df6a.out
### output ###
# +++ brew --prefix openblas
# ++ PKG_CONFIG_PATH=/opt/homebrew/opt/openblas/lib/pkgconfig:/opt/homebrew/opt/openblas/lib/pkgconfig:/opt/X11/lib/pkgconfig:/usr/X11/lib/pkgconfig
# ++ pkg-config --cflags openblas
# +++ brew --prefix openblas
# ++ PKG_CONFIG_PATH=/opt/homebrew/opt/openblas/lib/pkgconfig:/opt/homebrew/opt/openblas/lib/pkgconfig:/opt/X11/lib/pkgconfig:/usr/X11/lib/pkgconfig
# ++ pkg-config --libs openblas
# + cc -I/opt/homebrew/Cellar/openblas/0.3.29/include -fopenmp test.c -L/opt/homebrew/Cellar/openblas/0.3.29/lib -lopenblas
# clang: error: unsupported option '-fopenmp'
# clang: error: unsupported option '-fopenmp'

apparently this comes up in other places too, and https://thelinuxnotes.com/index.php/resolving-clang-error-unsupported-option-fopenmp-on-macos/ suggests installing a different llvm than shipped with the os. i'm hoping there is a simpler solution?

[nilsbecker]

any insights? for all i can tell, currently openblas and therefore owl are not installable on macos

[bord-o]

@nilsbecker I'm running into the same problem. Have you posted on the ocaml forum? If not I can make a post and maybe someone will have an idea.

Oh and I've tried and failed to get it compiling with another llvm install, I think because something isn't using the exported environment variables.

[mseri]

It seems to work fine with macports and with manual installs, I have just tested it locally.

From the look of it, this is a problem with hoembrew. Not the first time changes in hoembrew break the conf packages unfortunately. The the -fopenmp flag comes from the pkgconfig configuration shipped by homebrew's openblas.

I think the fix should be done in homebrew but I don't know where to start. Do you know if there is a fortran package and if installing that makes a difference?

[bord-o]

Not super familiar with BLAS or the mac ecosystem so I’m probably not the one to push that. Thanks for the heads up about macports though, I was able to get it working.

[nilsbecker]

interesting, that's a lead to follow i think. i found this: OpenMathLib/OpenBLAS#5156 (with a follow up discussion thread and related PR OpenMathLib/OpenBLAS#5180 )

i don't understand this fully -- to me it seems the easy solution would be if standard homebrew openblas would just not enable the broken OPENMP=1 flag, here: https://github.com/Homebrew/homebrew-core/blob/HEAD/Formula/o/openblas.rb

[mdales]

Just some notes on this incase it saves someone else so pain. You can work around this by switching to the home-brew llvm, but if your ultimate aim is to build owl, as mine was, then you'll hit other errors down the line.

I imagine you can play around with setting $CC, but here's what I did, and where it got me:

First I set PATH to have /opt/homebrew/opt/llvm/bin/ on it. OCaml calls cc, and there's no cc in that dir so I ended up making a symlink in there myself:

$ ln -s /opt/homebrew/opt/llvm/bin/clang /opt/homebrew/opt/llvm/cc

Once I did this, conf-openblas compiles fine.

However, Owl will then fail to link due to missing symbols, which also might be the case I guess if you try to use conf-openblas in anything else:

(cd _build/default && /Users/michael/.opam/5.3.0/bin/ocamlopt.opt -w -40 -g -o bin/owl_bin_viz.exe /Users/michael/.opam/5.3.0/lib/stdlib-shims/stdlib_shims.cmxa /Users/michael/.opam/5.3.0/lib/integers/integers.cmxa -I /Users/michael/.opam/5.3.0/lib/integers /Users/michael/.opam/5.3.0/lib/bigarray-compat/bigarray_compat.cmxa /Users/michael/.opam/5.3.0/lib/ctypes/ctypes.cmxa -I /Users/michael/.opam/5.3.0/lib/ctypes /Users/michael/.opam/5.3.0/lib/ocaml/str/str.cmxa /Users/michael/.opam/5.3.0/lib/ctypes/stubs/ctypes_stubs.cmxa /Users/michael/.opam/5.3.0/lib/ocaml/unix/unix.cmxa /Users/michael/.opam/5.3.0/lib/owl-base/owl_base.cmxa /Users/michael/.opam/5.3.0/lib/zip/zip.cmxa -I /Users/michael/.opam/5.3.0/lib/zip /Users/michael/.opam/5.3.0/lib/npy/npy.cmxa src/owl/owl.cmxa -I src/owl bin/.owl_bin_viz.eobjs/native/dune__exe__Owl_bin_viz.cmx -linkall)
ld: warning: ignoring duplicate libraries: '-lc++'
Undefined symbols for architecture arm64:
  "___kmpc_for_static_fini", referenced from:
      _c_float32_ndarray_get_slice_2.omp_outlined in libowl_stubs.a[66](owl_slicing_basic_stub.o)
      _c_float32_ndarray_get_slice_3.omp_outlined in libowl_stubs.a[66](owl_slicing_basic_stub.o)
      _c_float32_ndarray_get_slice_4.omp_outlined in libowl_stubs.a[66](owl_slicing_basic_stub.o)
      _c_float64_ndarray_get_slice_2.omp_outlined in libowl_stubs.a[66](owl_slicing_basic_stub.o)
      _c_float64_ndarray_get_slice_3.omp_outlined in libowl_stubs.a[66](owl_slicing_basic_stub.o)
      _c_float64_ndarray_get_slice_4.omp_outlined in libowl_stubs.a[66](owl_slicing_basic_stub.o)
      _c_complex32_ndarray_get_slice_2.omp_outlined in libowl_stubs.a[66](owl_slicing_basic_stub.o)
      ...
  "___kmpc_for_static_init_4", referenced from:
      _stub_float32_ndarray_maxpool_spatial_native.omp_outlined in libowl_stubs.a[73](owl_ndarray_pool_stub.o)
      _stub_float32_ndarray_maxpool_spatial_backward_native.omp_outlined in libowl_stubs.a[73](owl_ndarray_pool_stub.o)
      _stub_float32_ndarray_maxpool_cuboid_native.omp_outlined in libowl_stubs.a[73](owl_ndarray_pool_stub.o)
      _stub_float32_ndarray_maxpool_cuboid_backward_native.omp_outlined in libowl_stubs.a[73](owl_ndarray_pool_stub.o)
      _stub_float32_ndarray_maxpool_spatial_arg_native.omp_outlined in libowl_stubs.a[73](owl_ndarray_pool_stub.o)
      _stub_float32_ndarray_avgpool_spatial_native.omp_outlined in libowl_stubs.a[73](owl_ndarray_pool_stub.o)
      _stub_float32_ndarray_avgpool_spatial_backward_native.omp_outlined in libowl_stubs.a[73](owl_ndarray_pool_stub.o)
      ...
  "___kmpc_for_static_init_8", referenced from:
      _c_float32_ndarray_get_slice_2.omp_outlined in libowl_stubs.a[66](owl_slicing_basic_stub.o)
      _c_float32_ndarray_get_slice_3.omp_outlined in libowl_stubs.a[66](owl_slicing_basic_stub.o)
      _c_float32_ndarray_get_slice_4.omp_outlined in libowl_stubs.a[66](owl_slicing_basic_stub.o)
      _c_float64_ndarray_get_slice_2.omp_outlined in libowl_stubs.a[66](owl_slicing_basic_stub.o)
      _c_float64_ndarray_get_slice_3.omp_outlined in libowl_stubs.a[66](owl_slicing_basic_stub.o)
      _c_float64_ndarray_get_slice_4.omp_outlined in libowl_stubs.a[66](owl_slicing_basic_stub.o)
      _c_complex32_ndarray_get_slice_2.omp_outlined in libowl_stubs.a[66](owl_slicing_basic_stub.o)
      ...
  "___kmpc_fork_call", referenced from:
      _c_float32_ndarray_get_slice_2 in libowl_stubs.a[66](owl_slicing_basic_stub.o)
      _c_float32_ndarray_get_slice_3 in libowl_stubs.a[66](owl_slicing_basic_stub.o)
      _c_float32_ndarray_get_slice_4 in libowl_stubs.a[66](owl_slicing_basic_stub.o)
      _c_float32_ndarray_get_slice in libowl_stubs.a[66](owl_slicing_basic_stub.o)
      _c_float64_ndarray_get_slice_2 in libowl_stubs.a[66](owl_slicing_basic_stub.o)
      _c_float64_ndarray_get_slice_3 in libowl_stubs.a[66](owl_slicing_basic_stub.o)
      _c_float64_ndarray_get_slice_4 in libowl_stubs.a[66](owl_slicing_basic_stub.o)
      ...
ld: symbol(s) not found for architecture arm64
cc: error: linker command failed with exit code 1 (use -v to see invocation)
File "caml_startup", line 1:
Error: Error during linking (exit code 1)

I did try creating a new switch whereby I used the llvm from homebrew for everything, but that didn't work (at least with OCaml 5.3.0, as base 0.17 failed to build.

#=== ERROR while compiling base.v0.17.1 =======================================#
# context     2.3.0 | macos/arm64 | ocaml-base-compiler.5.3.0 | https://opam.ocaml.org#4d8fa0fb8fce3b6c8b06f29ebcfa844c292d4f3e
# path        ~/.opam/owl/.opam-switch/build/base.v0.17.1
# command     ~/.opam/opam-init/hooks/sandbox.sh build dune build -p base -j 15
# exit-code   1
# env-file    ~/.opam/log/base-41783-b4e95e.env
# output-file ~/.opam/log/base-41783-b4e95e.out
### output ###
# [...]
# (cd _build/default/src && /opt/homebrew/opt/llvm/bin/cc -O2 -fno-strict-aliasing -fwrapv -pthread -D_FILE_OFFSET_BITS=64 -fdiagnostics-color=always -D_LARGEFILE64_SOURCE -mpopcnt -g -I /Users/michael/.opam/owl/lib/ocaml -I /Users/michael/.opam/owl/lib/ocaml_intrinsics_kernel -I /Users/michael/.opam/owl/lib/sexplib0 -I ../hash_types/src -I ../shadow-stdlib/src -o bytes_stubs.o -c bytes_stubs.c)
# cc: error: unsupported option '-mpopcnt' for target 'arm64-apple-darwin24.3.0'
# File "src/dune", line 19, characters 67-77:
# 19 |   (names bytes_stubs exn_stubs int_math_stubs hash_stubs obj_stubs am_testing)
#                                                                         ^^^^^^^^^^
# (cd _build/default/src && /opt/homebrew/opt/llvm/bin/cc -O2 -fno-strict-aliasing -fwrapv -pthread -D_FILE_OFFSET_BITS=64 -fdiagnostics-color=always -D_LARGEFILE64_SOURCE -mpopcnt -g -I /Users/michael/.opam/owl/lib/ocaml -I /Users/michael/.opam/owl/lib/ocaml_intrinsics_kernel -I /Users/michael/.opam/owl/lib/sexplib0 -I ../hash_types/src -I ../shadow-stdlib/src -o am_testing.o -c am_testing.c)
# cc: error: unsupported option '-mpopcnt' for target 'arm64-apple-darwin24.3.0'
# (cd _build/default && /Users/michael/.opam/owl/bin/ocamlopt.opt -w -40 -g -I src/.base.objs/byte -I src/.base.objs/native -I /Users/michael/.opam/owl/lib/ocaml_intrinsics_kernel -I /Users/michael/.opam/owl/lib/sexplib0 -I hash_types/src/.base_internalhash_types.objs/byte -I hash_types/src/.base_internalhash_types.objs/native -I shadow-stdlib/src/.shadow_stdlib.objs/byte -I shadow-stdlib/src/.[...]
# File "src/array.ml", line 399, characters 28-75:
# 399 |       result := Bool.select ((keep_left_if [@inlined]) (compare x !result)) x !result
#                                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
# Warning 55 [inlining-impossible]: Cannot inline: Unknown function

Ultimately, not a successful endeavour, but I just wanted to document it here incase it saved someone else the effort of going down that avenue.

[dimpase]

upstream has merged a cmake-based build fix which allows one to impose the OpenMP library to use, see OpenMathLib/OpenBLAS#5180

[derekchiang]

For anyone who wanted a quick fix on Macbooks, you can literally modify /opt/homebrew/opt/openblas/lib/pkgconfig/openblas.pc and remove the line that says omp_opt=-fopenmp and OpenBLAS will compile successfully. Of course, this means you lose multi-threading, but all features should still work since OpenBLAS will just fall back to single-thread mode.

[dimpase]

Why would one need OpenBLAS on a modern macOS? Apple's Accelerate framework supports recent BLAS/CBLAS/LAPACK standards quite well (it wasn't the case until few years ago, but nowadays it's quite good).

[dimpase]

For anyone who wanted a quick fix on Macbooks, you can literally modify /opt/homebrew/opt/openblas/lib/pkgconfig/openblas.pc and remove the line that says omp_opt=-fopenmp and OpenBLAS will compile successfully. Of course, this means you lose multi-threading, but all features should still work since OpenBLAS will just fall back to single-thread mode.

as far as I understand, the configuration goes via pkg-config, and so to switch to Accelerate framework one can simply install somewhere a fake openblas.pc, as shown below, and make sure its location comes before the one for the real openblas in PKG_CONFIG_PATH.

Name: blas 
Description: Apple blas
Version: 42.0
URL: https://apple.com
Cflags:
Libs: -framework Accelerate