delocate-wheel and non-fixed brew dependency

[hredestig]

I am trying to use multibuild (great project btw, thanks a lot for this one) to make wheels for swiglpk but have difficulty to fix the library dependencies for the mac wheels. A reproducible example for is

$ git clone https://github.com/biosustain/swiglpk
$ brew install glpk
$ cd swiglpk/
$ python setup.py bdist_wheel
$ delocate-listdeps dist/swiglpk-1.2.22+4.g954dc63-cp35-cp35m-macosx_10_6_intel.whl 
/usr/local/Cellar/glpk/4.60/lib/libglpk.40.dylib
$ delocate-wheel -v -w fixed-wheel dist/swiglpk-1.2.22+4.g954dc63-cp35-cp35m-macosx_10_6_intel
$ delocate-listdeps fixed-wheel/swiglpk-1.2.22+4.g954dc63-cp35-cp35m-macosx_10_6_intel.whl 
/usr/local/Cellar/glpk/4.60/lib/libglpk.40.dylib

With the last call to delocate-listdeps I expected no output indicating that delocate-wheel had copied the dependency to the wheel. Am I missing something or is this a bug somehow?

[matthew-brett]

Does the wheel in fact contain the library? I seem to remember that delocate does not by default print out the libraries it copied, perhaps I should change that line in multibuild to avoid the confusion.

[hredestig]

no, pardon the confusion, the fixed wheel doesn't contain the library so something is going wrong when I do delocate-wheel

[matthew-brett]

Pardon me for not reading your message with enough attention. I'll have a look later on today.

[matthew-brett]

Ah - yes - I see. The wheel processor specifically looks in the subdirectories within the wheel, and not in the compiled modules at the top level. The reason is that it wants to create a new subdirectory, called .libs with the copied libraries in. This is unique to the package if it can work in a subdirectory (.../site-packages/yourpkg/.libs/copied_lib.dylib), but may well not be unique for a module at the top level (.../site-packages/.libs/copied_lib.dylib). So, to make it easier for delocate, you could make the module into its own package. Otherwise, I guess I'd need to work out some naming scheme for the generated lib / lib directory.

[hredestig]

Alright, thanks that makes sense, will try to adjust on our end

[phantomas1234]

Ok, I converted swiglpk into a package instead of a top level module (which was not straightforward at all since it uses swig) but now I get this when I run delocate-wheel -v -w fixed_wheels dist/swiglpk-1.3.1+7.gca90bd5.dirty-cp27-cp27m-macosx_10_5_x86_64.whl:

Fixing: dist/swiglpk-1.3.1+7.gca90bd5.dirty-cp27-cp27m-macosx_10_5_x86_64.whl
Traceback (most recent call last):
  File "/Users/niso/anaconda/envs/swiglpk2.7/bin/delocate-wheel", line 79, in <module>
    main()
  File "/Users/niso/anaconda/envs/swiglpk2.7/bin/delocate-wheel", line 71, in main
    check_verbose=opts.verbose)
  File "/Users/niso/anaconda/envs/swiglpk2.7/lib/python2.7/site-packages/delocate/delocating.py", line 377, in delocate_wheel
    lib_filt_func, copy_filt_func)
  File "/Users/niso/anaconda/envs/swiglpk2.7/lib/python2.7/site-packages/delocate/delocating.py", line 284, in delocate_path
    lib_dict = tree_libs(tree_path, lib_filt_func)
  File "/Users/niso/anaconda/envs/swiglpk2.7/lib/python2.7/site-packages/delocate/libsana.py", line 54, in tree_libs
    for install_name in get_install_names(depending_libpath):
  File "/Users/niso/anaconda/envs/swiglpk2.7/lib/python2.7/site-packages/delocate/tools.py", line 166, in get_install_names
    out = back_tick(['otool', '-L', filename])
  File "/Users/niso/anaconda/envs/swiglpk2.7/lib/python2.7/site-packages/delocate/tools.py", line 58, in back_tick
    cmd_str, retcode, err.decode('latin-1')))
RuntimeError: otool -L /private/var/folders/7q/y2z2wt4160g1n0yjvqq00ry8yzrbwb/T/tmp0_qWUG/wheel/swiglpk/__init__.py returned code 1 with error /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/objdump: '/private/var/folders/7q/y2z2wt4160g1n0yjvqq00ry8yzrbwb/T/tmp0_qWUG/wheel/swiglpk/__init__.py': The file was not recognized as a valid object file.
fatal error: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/otool: internal objdump command failed

Why would delocate try to inspeact a__init__.py file? I've uploaded the wheel to my dropbox: https://www.dropbox.com/s/8mxvnn6e4eii2lh/swiglpk-1.3.1%2B7.gca90bd5.dirty-cp27-cp27m-macosx_10_5_x86_64.whl?dl=1

It works fine otherwise (so the package structure is ok I hope). https://github.com/biosustain/swiglpk/tree/new_appveyor_config is what I used to create the wheel (using python setup.py bdist_wheel).

[matthew-brett]

Puzzling - I just cloned your repo, built a wheel and ran delocate on it:

pip install delocate
pip wheel  .
delocate-wheel -v swiglpk-1.3.3rc1-cp27-cp27m-macosx_10_6_intel.whl 
Fixing: swiglpk-1.3.3rc1-cp27-cp27m-macosx_10_6_intel.whl
Copied to package .dylibs directory:
  /usr/local/Cellar/glpk/4.60/lib/libglpk.40.dylib
  /usr/local/Cellar/gmp/6.1.1/lib/libgmp.10.dylib

I think you've run into problems with delocate and the latest OSX. Try the current github master version of delocate - does that fix it?

[matthew-brett]

I just uploaded a new release of delocate with the OSX 10.12 fixes - can you test?

[phantomas1234]

Indeed, I was already wondering why this was not causing any troubles on travis-ci. The new release fixed also the issue on macOS Sierra. Also it looks like making the swiglpk a package fixed the original we had. Thanks for your help 👍 you can close this issue if you like.

[matthew-brett]

Great - thanks for the quick feedback.