Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

try unbreaking libtiff on windows with new build #970

Merged
merged 1 commit into from
May 2, 2020

Conversation

visr
Copy link
Contributor

@visr visr commented May 2, 2020

Directly bumps to the latest 4.1.0 release from November 2019.

It seems the current libtiff build is broken on Windows, possibly since #961.

julia> using Libtiff_jll
ERROR: InitError: could not load library "C:\Users\visser_mn\.julia\artifacts\f98adb5b4de5c29a8d77def412b792beaec6180b\bin\libtiff-5.dll"
The specified module could not be found.
Stacktrace:
 [1] dlopen(::String, ::UInt32; throw_error::Bool) at D:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.4\Libdl\src\Libdl.jl:109
 [2] dlopen at D:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.4\Libdl\src\Libdl.jl:109 [inlined] (repeats 2 times)
 [3] __init__() at C:\Users\visser_mn\.julia\packages\Libtiff_jll\0kDRi\src\wrappers\x86_64-w64-mingw32.jl:45
 [4] _include_from_serialized(::String, ::Array{Any,1}) at .\loading.jl:697
 [5] _require_search_from_serialized(::Base.PkgId, ::String) at .\loading.jl:781
 [6] _require(::Base.PkgId) at .\loading.jl:1006
 [7] require(::Base.PkgId) at .\loading.jl:927
 [8] require(::Module, ::Symbol) at .\loading.jl:922
 [9] eval(::Module, ::Any) at .\boot.jl:331
 [10] eval_user_input(::Any, ::REPL.REPLBackend) at D:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.4\REPL\src\REPL.jl:86
 [11] run_backend(::REPL.REPLBackend) at C:\Users\visser_mn\.julia\packages\Revise\WkyNB\src\Revise.jl:1023
 [12] top-level scope at none:0
during initialization of module Libtiff_jll

I tried to debug using DependencyWalker (System32 dlls left out below):

julia> using DependencyWalker
[ Info: Precompiling DependencyWalker [7db386b5-e858-4964-8eb4-2fa66f8df882]

julia> Library(raw"C:\Users\visser_mn\.julia\artifacts\f98adb5b4de5c29a8d77def412b792beaec6180b\bin\libtiff-5.dll")
◼ C:\Users\visser_mn\.julia\artifacts\f98adb5b4de5c29a8d77def412b792beaec6180b\bin\libtiff-5.dll
  ◼ C:\Users\visser_mn\.julia\artifacts\217f5fb6408fcad8ec290be23f14646aab4e53b0\bin\libjpeg-62.dll
  ✗ libzstd.dll (NOT FOUND)
  ◼ C:\Users\visser_mn\.julia\artifacts\12dda53f058e2ad8360473e1df8d31d709724a38\bin\libz.dll

So it cannot find libzstd.dll. It appears to me the reason is that before it was not versioned, but now it is:

.julia\artifacts\2c987d0ee933c3900ce3f148b81d7f7c9c704c56\bin\libzstd.dll
BinaryBuilder.jl version: 0.2.2-888d6f2c4e049b9a277e79ddbfb2d50007ffa5a8

.julia\artifacts\eb77caa0e89a4b205fc8393d5aca496bfb13ea4e\bin\libzstd-1.dll
BinaryBuilder.jl version: 0.2.4-tree-e953eddbb5

Looking at the log of the latest Libtiff build, it seems to pick it up however:

Libtiff is now configured for x86_64-w64-mingw32
  ZSTD support:                       yes

But I guess that was built with the old unversioned libzstd? So perhaps we simply need to rebuild libtiff.
I guess we cannot use the package manager to require Zstd-v1.4.4+1.

From https://juliapackages.com/p/zstd_jll it seems the only other user of libstd is Blosc_jll, so perhaps that needs a rebuild as well.

Links
https://github.com/JuliaBinaryWrappers/Libtiff_jll.jl
https://github.com/JuliaBinaryWrappers/Zstd_jll.jl
https://github.com/JuliaBinaryWrappers/Blosc_jll.jl
JuliaGeo/Proj.jl#41
http://www.libtiff.org/build.html

Directly bumps to the latest 4.1.0 release from November 2019.

It seems the current libtiff build is broken on Windows, possibly since JuliaPackaging#961.

```
julia> using Libtiff_jll
ERROR: InitError: could not load library "C:\Users\visser_mn\.julia\artifacts\f98adb5b4de5c29a8d77def412b792beaec6180b\bin\libtiff-5.dll"
The specified module could not be found.
Stacktrace:
 [1] dlopen(::String, ::UInt32; throw_error::Bool) at D:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.4\Libdl\src\Libdl.jl:109
 [2] dlopen at D:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.4\Libdl\src\Libdl.jl:109 [inlined] (repeats 2 times)
 [3] __init__() at C:\Users\visser_mn\.julia\packages\Libtiff_jll\0kDRi\src\wrappers\x86_64-w64-mingw32.jl:45
 [4] _include_from_serialized(::String, ::Array{Any,1}) at .\loading.jl:697
 [5] _require_search_from_serialized(::Base.PkgId, ::String) at .\loading.jl:781
 [6] _require(::Base.PkgId) at .\loading.jl:1006
 [7] require(::Base.PkgId) at .\loading.jl:927
 [8] require(::Module, ::Symbol) at .\loading.jl:922
 [9] eval(::Module, ::Any) at .\boot.jl:331
 [10] eval_user_input(::Any, ::REPL.REPLBackend) at D:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.4\REPL\src\REPL.jl:86
 [11] run_backend(::REPL.REPLBackend) at C:\Users\visser_mn\.julia\packages\Revise\WkyNB\src\Revise.jl:1023
 [12] top-level scope at none:0
during initialization of module Libtiff_jll
```

I tried to debug using DependencyWalker (System32 dlls left out below):

```julia
julia> using DependencyWalker
[ Info: Precompiling DependencyWalker [7db386b5-e858-4964-8eb4-2fa66f8df882]

julia> Library(raw"C:\Users\visser_mn\.julia\artifacts\f98adb5b4de5c29a8d77def412b792beaec6180b\bin\libtiff-5.dll")
◼ C:\Users\visser_mn\.julia\artifacts\f98adb5b4de5c29a8d77def412b792beaec6180b\bin\libtiff-5.dll
  ◼ C:\Users\visser_mn\.julia\artifacts\217f5fb6408fcad8ec290be23f14646aab4e53b0\bin\libjpeg-62.dll
  ✗ libzstd.dll (NOT FOUND)
  ◼ C:\Users\visser_mn\.julia\artifacts\12dda53f058e2ad8360473e1df8d31d709724a38\bin\libz.dll
```

So it cannot find `libzstd.dll`. It appears to me the reason is that before it was not versioned, but now it is:

```
.julia\artifacts\2c987d0ee933c3900ce3f148b81d7f7c9c704c56\bin\libzstd.dll
BinaryBuilder.jl version: 0.2.2-888d6f2c4e049b9a277e79ddbfb2d50007ffa5a8

.julia\artifacts\eb77caa0e89a4b205fc8393d5aca496bfb13ea4e\bin\libzstd-1.dll
BinaryBuilder.jl version: 0.2.4-tree-e953eddbb5
```

Looking at the log of the latest Libtiff build, it seems to pick it up however:
```
Libtiff is now configured for x86_64-w64-mingw32
  ZSTD support:                       yes
```

But I guess that was built with the old unversioned `libzstd`? So perhaps we simply need to rebuild `libtiff`.
I guess we cannot use the package manager to require `Zstd-v1.4.4+1`.

From https://juliapackages.com/p/zstd_jll it seems the only other user of `libstd` is `Blosc_jll`, so perhaps that needs a rebuild as well.

__Links__
https://github.com/JuliaBinaryWrappers/Libtiff_jll.jl
https://github.com/JuliaBinaryWrappers/Zstd_jll.jl
https://github.com/JuliaBinaryWrappers/Blosc_jll.jl
JuliaGeo/Proj.jl#41
http://www.libtiff.org/build.html
visr added a commit to visr/Yggdrasil that referenced this pull request May 2, 2020
Explanation at JuliaPackaging#970. I just tried

```
(Proj4) pkg> add Blosc_jll
  Resolving package versions...
  Installed Lz4_jll ─── v1.9.2+0
  Installed Blosc_jll ─ v1.14.3+0
   Updating `C:\Users\visser_mn\.julia\dev\Proj4\Project.toml`
  [0b7ba130] + Blosc_jll v1.14.3+0
   Updating `C:\Users\visser_mn\.julia\dev\Proj4\Manifest.toml`
  [0b7ba130] + Blosc_jll v1.14.3+0
  [5ced341a] + Lz4_jll v1.9.2+0

julia> using Blosc_jll
[ Info: Precompiling Blosc_jll [0b7ba130-8d10-5ba8-a3d6-c5182647fed9]
ERROR: InitError: could not load library "C:\Users\visser_mn\.julia\artifacts\579a09774380e99d70add2ba8682e426c8d2d76e\bin\libblosc.dll"
The specified module could not be found.
Stacktrace:
 [1] dlopen(::String, ::UInt32; throw_error::Bool) at D:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.4\Libdl\src\Libdl.jl:109
 [2] dlopen at D:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.4\Libdl\src\Libdl.jl:109 [inlined] (repeats 2 times)
 [3] __init__() at C:\Users\visser_mn\.julia\packages\Blosc_jll\O9WPW\src\wrappers\x86_64-w64-mingw32-cxx11.jl:45
 [4] _include_from_serialized(::String, ::Array{Any,1}) at .\loading.jl:697
 [5] _require_from_serialized(::String) at .\loading.jl:748
 [6] _require(::Base.PkgId) at .\loading.jl:1039
 [7] require(::Base.PkgId) at .\loading.jl:927
 [8] require(::Module, ::Symbol) at .\loading.jl:922
 [9] eval(::Module, ::Any) at .\boot.jl:331
 [10] eval_user_input(::Any, ::REPL.REPLBackend) at D:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.4\REPL\src\REPL.jl:86
 [11] run_backend(::REPL.REPLBackend) at C:\Users\visser_mn\.julia\packages\Revise\WkyNB\src\Revise.jl:1023
 [12] top-level scope at none:0
during initialization of module Blosc_jll
```
@giordano
Copy link
Member

giordano commented May 2, 2020

Awesome investigation work (and use of DependencyWalker.jl 😁), thanks!

@giordano giordano merged commit fff46e6 into JuliaPackaging:master May 2, 2020
giordano pushed a commit that referenced this pull request May 2, 2020
Explanation at #970. I just tried

```
(Proj4) pkg> add Blosc_jll
  Resolving package versions...
  Installed Lz4_jll ─── v1.9.2+0
  Installed Blosc_jll ─ v1.14.3+0
   Updating `C:\Users\visser_mn\.julia\dev\Proj4\Project.toml`
  [0b7ba130] + Blosc_jll v1.14.3+0
   Updating `C:\Users\visser_mn\.julia\dev\Proj4\Manifest.toml`
  [0b7ba130] + Blosc_jll v1.14.3+0
  [5ced341a] + Lz4_jll v1.9.2+0

julia> using Blosc_jll
[ Info: Precompiling Blosc_jll [0b7ba130-8d10-5ba8-a3d6-c5182647fed9]
ERROR: InitError: could not load library "C:\Users\visser_mn\.julia\artifacts\579a09774380e99d70add2ba8682e426c8d2d76e\bin\libblosc.dll"
The specified module could not be found.
Stacktrace:
 [1] dlopen(::String, ::UInt32; throw_error::Bool) at D:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.4\Libdl\src\Libdl.jl:109
 [2] dlopen at D:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.4\Libdl\src\Libdl.jl:109 [inlined] (repeats 2 times)
 [3] __init__() at C:\Users\visser_mn\.julia\packages\Blosc_jll\O9WPW\src\wrappers\x86_64-w64-mingw32-cxx11.jl:45
 [4] _include_from_serialized(::String, ::Array{Any,1}) at .\loading.jl:697
 [5] _require_from_serialized(::String) at .\loading.jl:748
 [6] _require(::Base.PkgId) at .\loading.jl:1039
 [7] require(::Base.PkgId) at .\loading.jl:927
 [8] require(::Module, ::Symbol) at .\loading.jl:922
 [9] eval(::Module, ::Any) at .\boot.jl:331
 [10] eval_user_input(::Any, ::REPL.REPLBackend) at D:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.4\REPL\src\REPL.jl:86
 [11] run_backend(::REPL.REPLBackend) at C:\Users\visser_mn\.julia\packages\Revise\WkyNB\src\Revise.jl:1023
 [12] top-level scope at none:0
during initialization of module Blosc_jll
```
@visr visr deleted the patch-2 branch May 2, 2020 12:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants