Not working in heroku

[umlx5h]

Hi.
Can this gem work in heroku environment?
I use python 3.6, ruby 2.5.0 and rails 5.1.4 version in heroku.

I set PYCALL_DEBUG_FIND_LIBPYTHON=1 variable and got an following error.

Loading production environment (Rails 5.1.4)
irb(main):001:0> require 'pycall/import'
=> false
irb(main):002:0> include PyCall::Import
=> Object
irb(main):003:0> pyimport :math
DEBUG(find_libpython) find_libpython(".heroku/python/bin/python")
DEBUG(find_libpython) investigate_python_config(".heroku/python/bin/python")
DEBUG(find_libpython) libs: ["libpython3.6m.a", "libpython3.6m", "libpython3.6", "libpython"]
DEBUG(find_libpython) libpaths: ["/app/.heroku/python/lib", "/app/.heroku/python/lib", "", "/app/.heroku/python", "/app/.heroku/python/lib"]
DEBUG(find_libpython) Fiddle::DLError: /app/.heroku/python/lib/libpython3.6m.a: invalid ELF header
DEBUG(find_libpython) Unable to find /app/.heroku/python/lib/libpython3.6m.a.so
DEBUG(find_libpython) Unable to find /app/.heroku/python/lib/x86_64-linux-gnu/libpython3.6m.a
DEBUG(find_libpython) Unable to find /app/.heroku/python/lib/x86_64-linux-gnu/libpython3.6m.a.so
DEBUG(find_libpython) Fiddle::DLError: /app/.heroku/python/lib/libpython3.6m.a: invalid ELF header
DEBUG(find_libpython) Unable to find /app/.heroku/python/lib/libpython3.6m.a.so
DEBUG(find_libpython) Unable to find /app/.heroku/python/lib/x86_64-linux-gnu/libpython3.6m.a
DEBUG(find_libpython) Unable to find /app/.heroku/python/lib/x86_64-linux-gnu/libpython3.6m.a.so
DEBUG(find_libpython) Unable to find /libpython3.6m.a
DEBUG(find_libpython) Unable to find /libpython3.6m.a.so
DEBUG(find_libpython) Unable to find /x86_64-linux-gnu/libpython3.6m.a
DEBUG(find_libpython) Unable to find /x86_64-linux-gnu/libpython3.6m.a.so
DEBUG(find_libpython) Unable to find /app/.heroku/python/libpython3.6m.a
DEBUG(find_libpython) Unable to find /app/.heroku/python/libpython3.6m.a.so
DEBUG(find_libpython) Unable to find /app/.heroku/python/x86_64-linux-gnu/libpython3.6m.a
DEBUG(find_libpython) Unable to find /app/.heroku/python/x86_64-linux-gnu/libpython3.6m.a.so
DEBUG(find_libpython) Fiddle::DLError: /app/.heroku/python/lib/libpython3.6m.a: invalid ELF header
DEBUG(find_libpython) Unable to find /app/.heroku/python/lib/libpython3.6m.a.so
DEBUG(find_libpython) Unable to find /app/.heroku/python/lib/x86_64-linux-gnu/libpython3.6m.a
DEBUG(find_libpython) Unable to find /app/.heroku/python/lib/x86_64-linux-gnu/libpython3.6m.a.so
DEBUG(find_libpython) Unable to find /app/.heroku/python/lib/libpython3.6m
DEBUG(find_libpython) Unable to find /app/.heroku/python/lib/libpython3.6m.so
DEBUG(find_libpython) Unable to find /app/.heroku/python/lib/x86_64-linux-gnu/libpython3.6m
DEBUG(find_libpython) Unable to find /app/.heroku/python/lib/x86_64-linux-gnu/libpython3.6m.so
DEBUG(find_libpython) Unable to find /app/.heroku/python/lib/libpython3.6m
DEBUG(find_libpython) Unable to find /app/.heroku/python/lib/libpython3.6m.so
DEBUG(find_libpython) Unable to find /app/.heroku/python/lib/x86_64-linux-gnu/libpython3.6m
DEBUG(find_libpython) Unable to find /app/.heroku/python/lib/x86_64-linux-gnu/libpython3.6m.so
DEBUG(find_libpython) Unable to find /libpython3.6m
DEBUG(find_libpython) Unable to find /libpython3.6m.so
DEBUG(find_libpython) Unable to find /x86_64-linux-gnu/libpython3.6m
DEBUG(find_libpython) Unable to find /x86_64-linux-gnu/libpython3.6m.so
DEBUG(find_libpython) Unable to find /app/.heroku/python/libpython3.6m
DEBUG(find_libpython) Unable to find /app/.heroku/python/libpython3.6m.so
DEBUG(find_libpython) Unable to find /app/.heroku/python/x86_64-linux-gnu/libpython3.6m
DEBUG(find_libpython) Unable to find /app/.heroku/python/x86_64-linux-gnu/libpython3.6m.so
DEBUG(find_libpython) Unable to find /app/.heroku/python/lib/libpython3.6m
DEBUG(find_libpython) Unable to find /app/.heroku/python/lib/libpython3.6m.so
DEBUG(find_libpython) Unable to find /app/.heroku/python/lib/x86_64-linux-gnu/libpython3.6m
DEBUG(find_libpython) Unable to find /app/.heroku/python/lib/x86_64-linux-gnu/libpython3.6m.so
DEBUG(find_libpython) Unable to find /app/.heroku/python/lib/libpython3.6
DEBUG(find_libpython) Unable to find /app/.heroku/python/lib/libpython3.6.so
DEBUG(find_libpython) Unable to find /app/.heroku/python/lib/x86_64-linux-gnu/libpython3.6
DEBUG(find_libpython) Unable to find /app/.heroku/python/lib/x86_64-linux-gnu/libpython3.6.so
DEBUG(find_libpython) Unable to find /app/.heroku/python/lib/libpython3.6
DEBUG(find_libpython) Unable to find /app/.heroku/python/lib/libpython3.6.so
DEBUG(find_libpython) Unable to find /app/.heroku/python/lib/x86_64-linux-gnu/libpython3.6
DEBUG(find_libpython) Unable to find /app/.heroku/python/lib/x86_64-linux-gnu/libpython3.6.so
DEBUG(find_libpython) Unable to find /libpython3.6
DEBUG(find_libpython) Unable to find /libpython3.6.so
DEBUG(find_libpython) Unable to find /x86_64-linux-gnu/libpython3.6
DEBUG(find_libpython) Unable to find /x86_64-linux-gnu/libpython3.6.so
DEBUG(find_libpython) Unable to find /app/.heroku/python/libpython3.6
DEBUG(find_libpython) Unable to find /app/.heroku/python/libpython3.6.so
DEBUG(find_libpython) Unable to find /app/.heroku/python/x86_64-linux-gnu/libpython3.6
DEBUG(find_libpython) Unable to find /app/.heroku/python/x86_64-linux-gnu/libpython3.6.so
DEBUG(find_libpython) Unable to find /app/.heroku/python/lib/libpython3.6
DEBUG(find_libpython) Unable to find /app/.heroku/python/lib/libpython3.6.so
DEBUG(find_libpython) Unable to find /app/.heroku/python/lib/x86_64-linux-gnu/libpython3.6
DEBUG(find_libpython) Unable to find /app/.heroku/python/lib/x86_64-linux-gnu/libpython3.6.so
DEBUG(find_libpython) Unable to find /app/.heroku/python/lib/libpython
DEBUG(find_libpython) Unable to find /app/.heroku/python/lib/libpython.so
DEBUG(find_libpython) Unable to find /app/.heroku/python/lib/x86_64-linux-gnu/libpython
DEBUG(find_libpython) Unable to find /app/.heroku/python/lib/x86_64-linux-gnu/libpython.so
DEBUG(find_libpython) Unable to find /app/.heroku/python/lib/libpython
DEBUG(find_libpython) Unable to find /app/.heroku/python/lib/libpython.so
DEBUG(find_libpython) Unable to find /app/.heroku/python/lib/x86_64-linux-gnu/libpython
DEBUG(find_libpython) Unable to find /app/.heroku/python/lib/x86_64-linux-gnu/libpython.so
DEBUG(find_libpython) Unable to find /libpython
DEBUG(find_libpython) Unable to find /libpython.so
DEBUG(find_libpython) Unable to find /x86_64-linux-gnu/libpython
DEBUG(find_libpython) Unable to find /x86_64-linux-gnu/libpython.so
DEBUG(find_libpython) Unable to find /app/.heroku/python/libpython
DEBUG(find_libpython) Unable to find /app/.heroku/python/libpython.so
DEBUG(find_libpython) Unable to find /app/.heroku/python/x86_64-linux-gnu/libpython
DEBUG(find_libpython) Unable to find /app/.heroku/python/x86_64-linux-gnu/libpython.so
DEBUG(find_libpython) Unable to find /app/.heroku/python/lib/libpython
DEBUG(find_libpython) Unable to find /app/.heroku/python/lib/libpython.so
DEBUG(find_libpython) Unable to find /app/.heroku/python/lib/x86_64-linux-gnu/libpython
DEBUG(find_libpython) Unable to find /app/.heroku/python/lib/x86_64-linux-gnu/libpython.so
DEBUG(find_libpython) Fiddle::DLError: /app/.heroku/python/lib/libpython3.6m.a: invalid ELF header
DEBUG(find_libpython) Fiddle::DLError: libpython3.6m: cannot open shared object file: No such file or directory
DEBUG(find_libpython) Fiddle::DLError: libpython3.6: cannot open shared object file: No such file or directory
DEBUG(find_libpython) Fiddle::DLError: libpython: cannot open shared object file: No such file or directory
Traceback (most recent call last):
        1: from (irb):3
PyCall::PythonNotFound (PyCall::PythonNotFound)

Python can work by using command.

Loading production environment (Rails 5.1.4)
irb(main):001:0> system('python --version')
Python 3.6.4
=> true
irb(main):002:0> system('which python')
/app/.heroku/python/bin/python
=> true
irb(main):003:0> system('echo $PATH')
/app/vendor/bundle/ruby/2.5.0/bin:/app/bin:/app/vendor/bundle/bin:vendor/yarn-v1.0.2/bin/:/app/.heroku/python/bin:/usr/local/bin:/usr/bin:/bin
=> true
irb(main):004:0>

thanks.

[mrkn]

@umlx5h pycall.rb requires that Python is configured with --enable-shared.

But heroku's python seems not configured with --enable-shared.
See https://github.com/heroku/heroku-buildpack-python/blob/master/builds/runtimes/python-3.6.4#L13

[richgong]

I've forked it to see if this fixes the problem: https://github.com/richgong/heroku-buildpack-python

[raphaelfarzankashani]

Hi @mrkn - how can I access this repo and this link? https://github.com/heroku/heroku-buildpack-python/blob/master/builds/runtimes/python-3.6.4#L13

Any way to contact you directly?

[mrkn]

@raphaelfarzankashani I think @richgong's this commit fixes this issue.

Why do you want to contact me directly?

[mrkn]

I close this issue because this is not pycall.rb's responsibility.

[dsounded]

Hello, @mrkn and thank you for the gem

I am trying to make it work on Heroku and had no luck...

Still PyCall::PythonNotFound: PyCall::PythonNotFound

This is my fork which should patch --enable-shared issue.

dsounded/heroku-buildpack-python@43cf2de

Did anything change before this topic was discussed for the last time ?
It even can't start the Puma now because of that error

(works perfectly locally)

[dsounded]

Here is the output:

DEBUG(find_libpython) find_libpython(nil)
DEBUG(find_libpython) investigate_python_config("python3")
DEBUG(find_libpython) libs: ["libpython3.9.a", "libpython3.9", "libpython"]
DEBUG(find_libpython) libpaths: ["/app/.heroku/python/lib", "/app/.heroku/python/lib", "", "/app/.heroku/python", "/app/.heroku/python/lib"]
DEBUG(find_libpython) Fiddle::DLError: /app/.heroku/python/lib/libpython3.9.a: invalid ELF header
DEBUG(find_libpython) Unable to find /app/.heroku/python/lib/libpython3.9.a.so
DEBUG(find_libpython) Unable to find /app/.heroku/python/lib/x86_64-linux-gnu/libpython3.9.a
DEBUG(find_libpython) Unable to find /app/.heroku/python/lib/x86_64-linux-gnu/libpython3.9.a.so
DEBUG(find_libpython) Fiddle::DLError: /app/.heroku/python/lib/libpython3.9.a: invalid ELF header
DEBUG(find_libpython) Unable to find /app/.heroku/python/lib/libpython3.9.a.so
DEBUG(find_libpython) Unable to find /app/.heroku/python/lib/x86_64-linux-gnu/libpython3.9.a
DEBUG(find_libpython) Unable to find /app/.heroku/python/lib/x86_64-linux-gnu/libpython3.9.a.so
DEBUG(find_libpython) Unable to find /libpython3.9.a
DEBUG(find_libpython) Unable to find /libpython3.9.a.so
DEBUG(find_libpython) Unable to find /x86_64-linux-gnu/libpython3.9.a
DEBUG(find_libpython) Unable to find /x86_64-linux-gnu/libpython3.9.a.so
DEBUG(find_libpython) Unable to find /app/.heroku/python/libpython3.9.a
DEBUG(find_libpython) Unable to find /app/.heroku/python/libpython3.9.a.so
DEBUG(find_libpython) Unable to find /app/.heroku/python/x86_64-linux-gnu/libpython3.9.a
DEBUG(find_libpython) Unable to find /app/.heroku/python/x86_64-linux-gnu/libpython3.9.a.so
DEBUG(find_libpython) Fiddle::DLError: /app/.heroku/python/lib/libpython3.9.a: invalid ELF header
DEBUG(find_libpython) Unable to find /app/.heroku/python/lib/libpython3.9.a.so
DEBUG(find_libpython) Unable to find /app/.heroku/python/lib/x86_64-linux-gnu/libpython3.9.a
DEBUG(find_libpython) Unable to find /app/.heroku/python/lib/x86_64-linux-gnu/libpython3.9.a.so
DEBUG(find_libpython) Unable to find /app/.heroku/python/lib/libpython3.9
DEBUG(find_libpython) Unable to find /app/.heroku/python/lib/libpython3.9.so
DEBUG(find_libpython) Unable to find /app/.heroku/python/lib/x86_64-linux-gnu/libpython3.9
DEBUG(find_libpython) Unable to find /app/.heroku/python/lib/x86_64-linux-gnu/libpython3.9.so
DEBUG(find_libpython) Unable to find /app/.heroku/python/lib/libpython3.9
DEBUG(find_libpython) Unable to find /app/.heroku/python/lib/libpython3.9.so
DEBUG(find_libpython) Unable to find /app/.heroku/python/lib/x86_64-linux-gnu/libpython3.9
DEBUG(find_libpython) Unable to find /app/.heroku/python/lib/x86_64-linux-gnu/libpython3.9.so
DEBUG(find_libpython) Unable to find /libpython3.9
DEBUG(find_libpython) Unable to find /libpython3.9.so
DEBUG(find_libpython) Unable to find /x86_64-linux-gnu/libpython3.9
DEBUG(find_libpython) Unable to find /x86_64-linux-gnu/libpython3.9.so
DEBUG(find_libpython) Unable to find /app/.heroku/python/libpython3.9
DEBUG(find_libpython) Unable to find /app/.heroku/python/libpython3.9.so
DEBUG(find_libpython) Unable to find /app/.heroku/python/x86_64-linux-gnu/libpython3.9
DEBUG(find_libpython) Unable to find /app/.heroku/python/x86_64-linux-gnu/libpython3.9.so
DEBUG(find_libpython) Unable to find /app/.heroku/python/lib/libpython3.9
DEBUG(find_libpython) Unable to find /app/.heroku/python/lib/libpython3.9.so
DEBUG(find_libpython) Unable to find /app/.heroku/python/lib/x86_64-linux-gnu/libpython3.9
DEBUG(find_libpython) Unable to find /app/.heroku/python/lib/x86_64-linux-gnu/libpython3.9.so
DEBUG(find_libpython) Unable to find /app/.heroku/python/lib/libpython
DEBUG(find_libpython) Unable to find /app/.heroku/python/lib/libpython.so
DEBUG(find_libpython) Unable to find /app/.heroku/python/lib/x86_64-linux-gnu/libpython
DEBUG(find_libpython) Unable to find /app/.heroku/python/lib/x86_64-linux-gnu/libpython.so
DEBUG(find_libpython) Unable to find /app/.heroku/python/lib/libpython
DEBUG(find_libpython) Unable to find /app/.heroku/python/lib/libpython.so
DEBUG(find_libpython) Unable to find /app/.heroku/python/lib/x86_64-linux-gnu/libpython
DEBUG(find_libpython) Unable to find /app/.heroku/python/lib/x86_64-linux-gnu/libpython.so
DEBUG(find_libpython) Unable to find /libpython
DEBUG(find_libpython) Unable to find /libpython.so
DEBUG(find_libpython) Unable to find /x86_64-linux-gnu/libpython
DEBUG(find_libpython) Unable to find /x86_64-linux-gnu/libpython.so
DEBUG(find_libpython) Unable to find /app/.heroku/python/libpython
DEBUG(find_libpython) Unable to find /app/.heroku/python/libpython.so
DEBUG(find_libpython) Unable to find /app/.heroku/python/x86_64-linux-gnu/libpython
DEBUG(find_libpython) Unable to find /app/.heroku/python/x86_64-linux-gnu/libpython.so
DEBUG(find_libpython) Unable to find /app/.heroku/python/lib/libpython
DEBUG(find_libpython) Unable to find /app/.heroku/python/lib/libpython.so
DEBUG(find_libpython) Unable to find /app/.heroku/python/lib/x86_64-linux-gnu/libpython
DEBUG(find_libpython) Unable to find /app/.heroku/python/lib/x86_64-linux-gnu/libpython.so
DEBUG(find_libpython) Fiddle::DLError: /app/.heroku/python/lib/libpython3.9.a: invalid ELF header
DEBUG(find_libpython) Fiddle::DLError: libpython3.9: cannot open shared object file: No such file or directory
DEBUG(find_libpython) Fiddle::DLError: libpython: cannot open shared object file: No such file or directory
Traceback (most recent call last):
	4: from -e:1:in `<main>'
	3: from /app/vendor/bundle/ruby/2.7.0/gems/pycall-1.3.1/lib/pycall.rb:13:in `builtins'
	2: from /app/vendor/bundle/ruby/2.7.0/gems/pycall-1.3.1/lib/pycall/init.rb:16:in `const_missing'
	1: from /app/vendor/bundle/ruby/2.7.0/gems/pycall-1.3.1/lib/pycall/init.rb:35:in `init'
/app/vendor/bundle/ruby/2.7.0/gems/pycall-1.3.1/lib/pycall/libpython/finder.rb:95:in `find_libpython': PyCall::PythonNotFound (PyCall::PythonNotFound)

[mrkn]

The debug log showed that pycall couldn't find libpython.so in all the candidate locations.

Where did you install the python built with --enable-shared? You should specify the location of the appropriate python command in PYTHON environment variable.

[dsounded]

Yeah. actually wanted to leave a comment with the solution...
For Heroku it works with the fork I provided + you need to specify LIBPYTHON as /app/.heroku/python/bin/python3.9 in my case (I wanted to use 3.9.0) since default python and python3 execs are infrastructures ones.

To find needed executable path you can run heroku run whereis python that helped me to find a path for needed version and now it looks good.

Thank you again for your time and this library!
Maybe this comment will help someone in the future :)

[ferrisoxide]

I realise this issue is closed, but if it helps anyone I managed to get pycall working Heroku working using this buildpack:

https://github.com/ReforgeHQ/heroku-buildpack-python.git