Fix TensorFlow and PyTorch compatibility #3574

pcmoritz · 2018-12-19T01:50:27Z

What do these changes do?

Related issue number

pcmoritz · 2018-12-19T01:51:40Z

cmake/Modules/BoostExternalProject.cmake


-  set(Boost_TAR_GZ_URL http://dl.bintray.com/boostorg/release/1.65.1/source/boost_1_65_1.tar.gz)
+  set(Boost_TAR_GZ_URL http://dl.bintray.com/boostorg/release/1.69.0/source/boost_1_69_0.tar.gz)


Upgrading boost was needed to fix some incompatibility with macOS

…-compat

AmplabJenkins · 2018-12-21T03:58:31Z

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/10246/
Test FAILed.

AmplabJenkins · 2018-12-21T09:42:59Z

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/10253/
Test FAILed.

AmplabJenkins · 2018-12-21T09:57:46Z

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/10254/
Test FAILed.

pcmoritz · 2018-12-21T16:45:48Z

jenkins retest this please

AmplabJenkins · 2018-12-21T18:55:53Z

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/10266/
Test FAILed.

pcmoritz · 2018-12-21T21:04:11Z

jenkins retest this please

AmplabJenkins · 2018-12-21T23:17:22Z

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/10269/
Test FAILed.

robertnishihara · 2018-12-22T01:29:23Z

python/README-building-wheels.md

@@ -9,7 +9,7 @@ produce .whl files owned by root.
 Inside the root directory (i.e., one level above this python directory), run

 ```
-docker run --rm -w /ray -v `pwd`:/ray -ti quay.io/xhochy/arrow_manylinux1_x86_64_base:latest /ray/python/build-wheel-manylinux1.sh
+docker run --rm -w /ray -v `pwd`:/ray -ti rayproject/ray_arrow_manylinux1_x86_64_base:latest /ray/python/build-wheel-manylinux1.sh


The script for building this docker image needs to be committed somewhere and documented here.

I think this is not needed any more, I'm trying to remove it.

AmplabJenkins · 2018-12-22T02:13:45Z

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/10275/
Test FAILed.

AmplabJenkins · 2018-12-22T05:37:40Z

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/10282/
Test FAILed.

robertnishihara · 2018-12-22T07:35:11Z

cmake/Modules/ArrowExternalProject.cmake

@@ -14,11 +14,11 @@
 #  - PLASMA_STATIC_LIB
 #  - PLASMA_SHARED_LIB

-set(arrow_URL https://github.com/apache/arrow.git)
-# The PR for this commit is https://github.com/apache/arrow/pull/3154. We
+set(arrow_URL https://github.com/ray-project/arrow.git)


Can you document why we're cloning a fork and maybe include a link to the discussion in the arrow PR?

robertnishihara · 2018-12-22T07:35:47Z

cmake/Modules/ThirdpartyToolchain.cmake

@@ -54,11 +54,14 @@ ADD_THIRDPARTY_LIB(boost_system
  STATIC_LIB ${Boost_SYSTEM_LIBRARY})
 ADD_THIRDPARTY_LIB(boost_filesystem
  STATIC_LIB ${Boost_FILESYSTEM_LIBRARY})
+  ADD_THIRDPARTY_LIB(boost_thread


fix indentation, same with above lines I think

robertnishihara · 2018-12-22T07:38:40Z

Looks good to me other than the comments I left and the comments discussed in person (that the diff from the main arrow repository should be squashed into a single commit).

AmplabJenkins · 2018-12-22T09:07:31Z

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/10290/
Test FAILed.

AmplabJenkins · 2018-12-22T10:02:27Z

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/10291/
Test FAILed.

pcmoritz added 27 commits December 15, 2018 21:09

remove tensorflow workaround

6301f8f

update docker

a347461

update

e61516a

add boost threads

6b6d59c

update

66908aa

add date_time, too

106608e

more flags

95ba9bc

update

0114b9b

update

5c393f7

update

84ff57e

fix

6c34543

update

ef4ec90

update

e1b25eb

update

785c73d

update

8d0b97d

update

6286371

fix

3ad0246

fix

4044408

update

01137ee

fix

13d5a25

fix

d1cc6ed

fix

77212ef

update

c6ac872

update

f8b75ef

update

9534cca

fix

60888b5

Merge branch 'master' into ray-tf-compat

d5e1f08

pcmoritz commented Dec 19, 2018

View reviewed changes

pcmoritz added 2 commits December 18, 2018 18:15

fix

d19990e

Merge branch 'ray-tf-compat' of github.com:pcmoritz/ray-1 into ray-tf…

2ec4a63

…-compat

update

7958751

pcmoritz force-pushed the ray-tf-compat branch from 12af5c3 to 7958751 Compare December 21, 2018 03:31

pcmoritz added 2 commits December 20, 2018 23:34

update

19f1ecb

update

85b88da

Merge branch 'master' into ray-tf-compat

75856ad

rerun Travis

04f1045

robertnishihara reviewed Dec 22, 2018

View reviewed changes

fix

a516d38

pcmoritz changed the title ~~[WIP] Fix TensorFlow and PyTorch compatibility~~ Fix TensorFlow and PyTorch compatibility Dec 22, 2018

robertnishihara reviewed Dec 22, 2018

View reviewed changes

pcmoritz added 2 commits December 21, 2018 23:46

cosmetics

f822d04

fix

79bac41

atumanov approved these changes Dec 22, 2018

View reviewed changes

atumanov merged commit e578a38 into ray-project:master Dec 22, 2018

atumanov deleted the ray-tf-compat branch December 22, 2018 21:26

robertnishihara mentioned this pull request Dec 24, 2018

Segmentation fault with ray 0.6.0 and PyTorch 0.4.1 and 1.0.0 #3520

Closed

guoyuhong mentioned this pull request Dec 24, 2018

Fix new boost libs failure in cache-lib mode and add test to cover collect_dependent_libs.sh #3627

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Fix TensorFlow and PyTorch compatibility #3574

Fix TensorFlow and PyTorch compatibility #3574

pcmoritz commented Dec 19, 2018 •

edited

Loading

pcmoritz Dec 19, 2018

AmplabJenkins commented Dec 21, 2018

AmplabJenkins commented Dec 21, 2018

AmplabJenkins commented Dec 21, 2018

pcmoritz commented Dec 21, 2018

AmplabJenkins commented Dec 21, 2018

pcmoritz commented Dec 21, 2018

AmplabJenkins commented Dec 21, 2018

robertnishihara Dec 22, 2018

pcmoritz Dec 22, 2018

AmplabJenkins commented Dec 22, 2018

AmplabJenkins commented Dec 22, 2018

robertnishihara Dec 22, 2018

robertnishihara Dec 22, 2018

robertnishihara commented Dec 22, 2018

AmplabJenkins commented Dec 22, 2018

AmplabJenkins commented Dec 22, 2018


		set(Boost_TAR_GZ_URL http://dl.bintray.com/boostorg/release/1.65.1/source/boost_1_65_1.tar.gz)
		set(Boost_TAR_GZ_URL http://dl.bintray.com/boostorg/release/1.69.0/source/boost_1_69_0.tar.gz)

Fix TensorFlow and PyTorch compatibility #3574

Fix TensorFlow and PyTorch compatibility #3574

Conversation

pcmoritz commented Dec 19, 2018 • edited Loading

What do these changes do?

Related issue number

pcmoritz Dec 19, 2018

Choose a reason for hiding this comment

AmplabJenkins commented Dec 21, 2018

AmplabJenkins commented Dec 21, 2018

AmplabJenkins commented Dec 21, 2018

pcmoritz commented Dec 21, 2018

AmplabJenkins commented Dec 21, 2018

pcmoritz commented Dec 21, 2018

AmplabJenkins commented Dec 21, 2018

robertnishihara Dec 22, 2018

Choose a reason for hiding this comment

pcmoritz Dec 22, 2018

Choose a reason for hiding this comment

AmplabJenkins commented Dec 22, 2018

AmplabJenkins commented Dec 22, 2018

robertnishihara Dec 22, 2018

Choose a reason for hiding this comment

robertnishihara Dec 22, 2018

Choose a reason for hiding this comment

robertnishihara commented Dec 22, 2018

AmplabJenkins commented Dec 22, 2018

AmplabJenkins commented Dec 22, 2018

pcmoritz commented Dec 19, 2018 •

edited

Loading