M1 Monterey: libblosc not defined

[cortner]

On Julia 1.7, on an M1, Monterey, I cannot use Blosc anymore. I ran into this problem trying to install HDF5 which had a similar error message but in that case I could simply point it to the home-brew installation of libhdf5. For Blosc I found no instruction how to do this.

Here are my system information and errors:

julia> versioninfo()
Julia Version 1.7.0-rc3.6
Commit 5dba09613d (2021-11-16 11:15 UTC)
Platform Info:
  OS: macOS (arm64-apple-darwin21.1.0)
  CPU: Apple M1 Max
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-12.0.1 (ORCJIT, cyclone)

(@v1.7) pkg> status
      Status `~/.julia/environments/v1.7/Project.toml`
  [a74b3585] Blosc v0.7.1
  [7073ff75] IJulia v1.23.2

After restarting Julia:

   _       _ _(_)_     |  Documentation: https://docs.julialang.org
  (_)     | (_) (_)    |
   _ _   _| |_  __ _   |  Type "?" for help, "]?" for Pkg help.
  | | | | | | |/ _` |  |
  | | |_| | | | (_| |  |  Version 1.7.0-rc3.6 (2021-11-16)
 _/ |\__'_|_|_|\__'_|  |  backports-release-1.7/5dba09613d (fork: 398 commits, 168 days)
|__/                   |

(@v1.7) pkg> build Blosc

julia> using Blosc
ERROR: InitError: UndefVarError: libblosc not defined
Stacktrace:
 [1] __init__()
   @ Blosc ~/.julia/packages/Blosc/vjmKP/src/Blosc.jl:6
 [2] _include_from_serialized(path::String, depmods::Vector{Any})
   @ Base ./loading.jl:768
 [3] _require_search_from_serialized(pkg::Base.PkgId, sourcepath::String)
   @ Base ./loading.jl:854
 [4] _require(pkg::Base.PkgId)
   @ Base ./loading.jl:1097
 [5] require(uuidkey::Base.PkgId)
   @ Base ./loading.jl:1013
 [6] require(into::Module, mod::Symbol)
   @ Base ./loading.jl:997
during initialization of module Blosc

[kylebeggs]

I am also having this issue. I tried adding libblosc to my path but perhaps I am still not doing something correctly.

[ajalexei]

I second this. The reason seems to be that the Blosc_jll package is not updated to support Darwin arm64 architecture: one needs to add the respective wrapper and the artifact, e.g. the libblosc.dylib.
The override mechanism is not working as well and I cannot point to a local installation of the libblosc.

[Aemmel]

Having the exact same issue. Did someone find a solution yet?
(The same issue appears when trying to install HDF5.jl)

[Aemmel]

I found a solution! It's really hacky and probably breaks stuff later on, but for now it works. The problem is, as ajalexei stated, that it can't find the blosc library, thus the ccall(:blosc_init, libblosc, ... call fails, since it does not know what libblosc is, and the wrapper Blosc_jll does not have it configured for mac ARM (I think). So my solution:

1. install c-blosc via brew install c-blosc
2. find the .dylib file under "/opt/homebrew/Cellar/c-blosc/1.21.1/lib/libblosc.1.21.1.dylib" (for my version)
3. go to ~.julia/packages/Blosc/vjmKP/src/Blosc.jl and add before the __init__() function: const libblosc="/opt/homebrew/Cellar/c-blosc/1.21.1/lib/libblosc.1.21.1.dylib"
4. Blosc now finds the libblosc file and compiles succesfully.

As I said, I am pretty sure this can break things later on and is probably not a good solution. But it seems to work for now.

[kylebeggs]

Seems like a good workaround for now. I also installed a forked HDF5.jl that does not load Blosc by default. This was suggested to me by @mkitti on Slack. I didn't realize Blosc is not always needed. So this may work for you as well, although it does break the MAT package for me due to version conflicts.

pkg> add https://github.com/JaneliaSciComp/HDF5.jl.git#janelia/bzip2_lz4_std_filters

[cortner]

can you post which HDF5 fork you are using please?

[mkitti]

I think he's talking about my fork at HHMI Janelia Scientific Computing, which is been submitted to HDF5.jl as JuliaIO/HDF5.jl#875 .

The main practical effect is that it drops use of Blosc as a default filter that is loaded by default.
https://github.com/JaneliaSciComp/HDF5.jl.git#janelia/bzip2_lz4_std_filters

I have slightly more advanced branch which goes further and actually drops the Blosc dependency from HDF5.jl completely:
https://github.com/JaneliaSciComp/HDF5.jl/tree/janelia/h5z_subdir_packages

This creates an independent subdirectory package called H5Zblosc which can be optionally loaded.

That has been submitted to HDF5.jl as JuliaIO/HDF5.jl#880.

I may merge the later branch into the former at some point.

I am also trying to address the Blosc 1 issue directly via JuliaPackaging/Yggdrasil#4052

[mkitti]

Blosc_jll should now be available for aarch64-apple-darwin as of version 1.21.1

https://github.com/JuliaBinaryWrappers/Blosc_jll.jl/releases/download/Blosc-v1.21.1%2B0/Blosc.v1.21.1.aarch64-apple-darwin.tar.gz

Can anyone confirm if this has been fixed?

[ajalexei]

Even if too late -- it does compile and work now, thanks