Error while installing from source

[kunalrdeshmukh]

System information

• OS Platform and Distribution (e.g., Linux Ubuntu 16.04): Ubuntu 18.04 LTS
• Ray installed from (source or binary): source
• Ray version: latest
• Python version: 2.7
• Exact command to reproduce: pip install -e . --verbose

Describe the problem

I faced error while installing ray library from source on ubuntu. I tried to replicate the issue on new VM and I am successful in doing it.

Source code / logs

PFA the output of above command it includes error as well.
installation.txt

[raulchen]

the error says
fatal: unable to access 'https://github.com/apache/arrow.git/': Could not resolve host: github.com
do you have network connection on the machine?

[kunalrdeshmukh]

Yes. in fact i had ran repo clone command on same computer minutes ago.

…

On Thu, Jun 14, 2018, 10:55 PM Hao Chen ***@***.***> wrote: the error says fatal: unable to access 'https://github.com/apache/arrow.git/': Could not resolve host: github.com do you have network connection on the machine? — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#2257 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AF8TkfaTnN2tlDEpHBf1i2erETfxwbU8ks5t80xqgaJpZM4UpB-o> .

[kunalrdeshmukh]

I tried to run again today. while ensuring good internet connectivity. but I got the error again.
PFA logs.
output.txt

[richardliaw]

This particular error looks like you need to clean the build from a previous installation; maybe try something like git clean -f -f -x -d and then rebuild?

[kunalrdeshmukh]

I tried it on brand new ubuntu VM and I tried git clean -f -f -x -d as well as suggested by you.
But I got same error every time.

[guoyuhong]

@richardliaw @robertnishihara I think the problem is that node_manager.cc has included local_scheduler_generated.h, but in the CMakeList.txt of that lib, this is not added as a dependency of the lib.

This problem will occur randomly because it is not guaranteed that gen_local_scheduler_fbs will always build ahead of raylib. In most cases, gen_local_scheduler_fbs will be built ahead of build raylib and the building process will success. I think the root cause is that raylet is design to decouple with old ray backend. However, in the node_manger.cc, there is a coupling of both parts to guarantee the enum value is right.

Is that correct?

[robertnishihara]

Please try with the latest master and let me know if the issue persists.

[kunalrdeshmukh]

Hi @robertnishihara
I got error again. It was some cython related error. so I installed cython and ran again.
PFA error I got after running it again
installation.txt
.

[robertnishihara]

You could always try installing Ray inside of Docker, since that should be reproducible. E.g., https://github.com/ray-project/ray/blob/master/build-docker.sh.

[kunalrdeshmukh]

i ran just 2 commands more than the ones suggested on http://ray.readthedocs.io/en/latest/installation.html

sudo apt-get install python-pip python-dev build-essential
and
pip install git

then

sudo apt-get update
sudo apt-get install -y cmake pkg-config build-essential autoconf curl libtool unzip flex bison python
sudo apt-get install python-dev # For Python 2.
sudo apt-get install python3-dev # For Python 3.

pip install cmake

pip install cython

git clone https://github.com/ray-project/ray.git
cd ray/python
pip install -e . --verbose

I ran the script suggested by you above. Installation was successful. It seems this installation uses conda. Can this be an issue?
Also, I got several errors while installing it. but it appears to run fine. PFA screenshot for your reference.

anyways, this will work for me so my problem is resolved. Thanks for your help.

[guoyuhong]

@robertnishihara @kunalrdeshmukh This time the error message changed... I think I found the root cause and I can reproduce this failure by change make -j${PARALLEL} to make -j1 in build.sh. I found @kunalrdeshmukh used this setting. My default setting will use make -j8. In multi-thread mode, some dependencies will be compiled by other threads. That is why the local_scheduler failure does not appear in most cases. I will try to fix problem once for all.