Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add zstdmt target for Zstd #961

Merged
merged 2 commits into from
May 1, 2020
Merged

Add zstdmt target for Zstd #961

merged 2 commits into from
May 1, 2020

Conversation

Keno
Copy link
Contributor

@Keno Keno commented May 1, 2020

We already build it, it just doesn't have an export in the jll.

We already build it, it just doesn't have an export in the jll
@giordano
Copy link
Member

giordano commented May 1, 2020

For some reasons, CMakeLists.txt wants to build zstdmt only for Unices, instead meson.build is happy to build it as long as multi-threading is enabled. I changed the build script to use meson instead of CMake

@giordano giordano merged commit c990191 into master May 1, 2020
@giordano giordano deleted the kf/zstdmt branch May 1, 2020 17:47
visr added a commit to visr/Yggdrasil that referenced this pull request May 2, 2020
Directly bumps to the latest 4.1.0 release from November 2019.

It seems the current libtiff build is broken on Windows, possibly since JuliaPackaging#961.

```
julia> using Libtiff_jll
ERROR: InitError: could not load library "C:\Users\visser_mn\.julia\artifacts\f98adb5b4de5c29a8d77def412b792beaec6180b\bin\libtiff-5.dll"
The specified module could not be found.
Stacktrace:
 [1] dlopen(::String, ::UInt32; throw_error::Bool) at D:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.4\Libdl\src\Libdl.jl:109
 [2] dlopen at D:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.4\Libdl\src\Libdl.jl:109 [inlined] (repeats 2 times)
 [3] __init__() at C:\Users\visser_mn\.julia\packages\Libtiff_jll\0kDRi\src\wrappers\x86_64-w64-mingw32.jl:45
 [4] _include_from_serialized(::String, ::Array{Any,1}) at .\loading.jl:697
 [5] _require_search_from_serialized(::Base.PkgId, ::String) at .\loading.jl:781
 [6] _require(::Base.PkgId) at .\loading.jl:1006
 [7] require(::Base.PkgId) at .\loading.jl:927
 [8] require(::Module, ::Symbol) at .\loading.jl:922
 [9] eval(::Module, ::Any) at .\boot.jl:331
 [10] eval_user_input(::Any, ::REPL.REPLBackend) at D:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.4\REPL\src\REPL.jl:86
 [11] run_backend(::REPL.REPLBackend) at C:\Users\visser_mn\.julia\packages\Revise\WkyNB\src\Revise.jl:1023
 [12] top-level scope at none:0
during initialization of module Libtiff_jll
```

I tried to debug using DependencyWalker (System32 dlls left out below):

```julia
julia> using DependencyWalker
[ Info: Precompiling DependencyWalker [7db386b5-e858-4964-8eb4-2fa66f8df882]

julia> Library(raw"C:\Users\visser_mn\.julia\artifacts\f98adb5b4de5c29a8d77def412b792beaec6180b\bin\libtiff-5.dll")
◼ C:\Users\visser_mn\.julia\artifacts\f98adb5b4de5c29a8d77def412b792beaec6180b\bin\libtiff-5.dll
  ◼ C:\Users\visser_mn\.julia\artifacts\217f5fb6408fcad8ec290be23f14646aab4e53b0\bin\libjpeg-62.dll
  ✗ libzstd.dll (NOT FOUND)
  ◼ C:\Users\visser_mn\.julia\artifacts\12dda53f058e2ad8360473e1df8d31d709724a38\bin\libz.dll
```

So it cannot find `libzstd.dll`. It appears to me the reason is that before it was not versioned, but now it is:

```
.julia\artifacts\2c987d0ee933c3900ce3f148b81d7f7c9c704c56\bin\libzstd.dll
BinaryBuilder.jl version: 0.2.2-888d6f2c4e049b9a277e79ddbfb2d50007ffa5a8

.julia\artifacts\eb77caa0e89a4b205fc8393d5aca496bfb13ea4e\bin\libzstd-1.dll
BinaryBuilder.jl version: 0.2.4-tree-e953eddbb5
```

Looking at the log of the latest Libtiff build, it seems to pick it up however:
```
Libtiff is now configured for x86_64-w64-mingw32
  ZSTD support:                       yes
```

But I guess that was built with the old unversioned `libzstd`? So perhaps we simply need to rebuild `libtiff`.
I guess we cannot use the package manager to require `Zstd-v1.4.4+1`.

From https://juliapackages.com/p/zstd_jll it seems the only other user of `libstd` is `Blosc_jll`, so perhaps that needs a rebuild as well.

__Links__
https://github.com/JuliaBinaryWrappers/Libtiff_jll.jl
https://github.com/JuliaBinaryWrappers/Zstd_jll.jl
https://github.com/JuliaBinaryWrappers/Blosc_jll.jl
JuliaGeo/Proj.jl#41
http://www.libtiff.org/build.html
giordano pushed a commit that referenced this pull request May 2, 2020
Directly bumps to the latest 4.1.0 release from November 2019.

It seems the current libtiff build is broken on Windows, possibly since #961.

```
julia> using Libtiff_jll
ERROR: InitError: could not load library "C:\Users\visser_mn\.julia\artifacts\f98adb5b4de5c29a8d77def412b792beaec6180b\bin\libtiff-5.dll"
The specified module could not be found.
Stacktrace:
 [1] dlopen(::String, ::UInt32; throw_error::Bool) at D:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.4\Libdl\src\Libdl.jl:109
 [2] dlopen at D:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.4\Libdl\src\Libdl.jl:109 [inlined] (repeats 2 times)
 [3] __init__() at C:\Users\visser_mn\.julia\packages\Libtiff_jll\0kDRi\src\wrappers\x86_64-w64-mingw32.jl:45
 [4] _include_from_serialized(::String, ::Array{Any,1}) at .\loading.jl:697
 [5] _require_search_from_serialized(::Base.PkgId, ::String) at .\loading.jl:781
 [6] _require(::Base.PkgId) at .\loading.jl:1006
 [7] require(::Base.PkgId) at .\loading.jl:927
 [8] require(::Module, ::Symbol) at .\loading.jl:922
 [9] eval(::Module, ::Any) at .\boot.jl:331
 [10] eval_user_input(::Any, ::REPL.REPLBackend) at D:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.4\REPL\src\REPL.jl:86
 [11] run_backend(::REPL.REPLBackend) at C:\Users\visser_mn\.julia\packages\Revise\WkyNB\src\Revise.jl:1023
 [12] top-level scope at none:0
during initialization of module Libtiff_jll
```

I tried to debug using DependencyWalker (System32 dlls left out below):

```julia
julia> using DependencyWalker
[ Info: Precompiling DependencyWalker [7db386b5-e858-4964-8eb4-2fa66f8df882]

julia> Library(raw"C:\Users\visser_mn\.julia\artifacts\f98adb5b4de5c29a8d77def412b792beaec6180b\bin\libtiff-5.dll")
◼ C:\Users\visser_mn\.julia\artifacts\f98adb5b4de5c29a8d77def412b792beaec6180b\bin\libtiff-5.dll
  ◼ C:\Users\visser_mn\.julia\artifacts\217f5fb6408fcad8ec290be23f14646aab4e53b0\bin\libjpeg-62.dll
  ✗ libzstd.dll (NOT FOUND)
  ◼ C:\Users\visser_mn\.julia\artifacts\12dda53f058e2ad8360473e1df8d31d709724a38\bin\libz.dll
```

So it cannot find `libzstd.dll`. It appears to me the reason is that before it was not versioned, but now it is:

```
.julia\artifacts\2c987d0ee933c3900ce3f148b81d7f7c9c704c56\bin\libzstd.dll
BinaryBuilder.jl version: 0.2.2-888d6f2c4e049b9a277e79ddbfb2d50007ffa5a8

.julia\artifacts\eb77caa0e89a4b205fc8393d5aca496bfb13ea4e\bin\libzstd-1.dll
BinaryBuilder.jl version: 0.2.4-tree-e953eddbb5
```

Looking at the log of the latest Libtiff build, it seems to pick it up however:
```
Libtiff is now configured for x86_64-w64-mingw32
  ZSTD support:                       yes
```

But I guess that was built with the old unversioned `libzstd`? So perhaps we simply need to rebuild `libtiff`.
I guess we cannot use the package manager to require `Zstd-v1.4.4+1`.

From https://juliapackages.com/p/zstd_jll it seems the only other user of `libstd` is `Blosc_jll`, so perhaps that needs a rebuild as well.

__Links__
https://github.com/JuliaBinaryWrappers/Libtiff_jll.jl
https://github.com/JuliaBinaryWrappers/Zstd_jll.jl
https://github.com/JuliaBinaryWrappers/Blosc_jll.jl
JuliaGeo/Proj.jl#41
http://www.libtiff.org/build.html
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants