Update apache arrow to include TensorFlow fix

[pcmoritz]

This is intended to fix #2159.

[robertnishihara]

There may be some plasma hashing related issues. We'll see when the tests run.

[robertnishihara]

#2319 also updates the Arrow commit. Can you take a look?

[AmplabJenkins]

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/6462/
Test PASSed.

[pcmoritz]

Yeah, the changes in the two PRs are mostly orthogonal (except updating the hash of course); this one is mainly incorporating the plasma API changes and the other one doesn't include that commit.

[AmplabJenkins]

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/6463/
Test FAILed.

[AmplabJenkins]

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/6471/
Test PASSed.

[guoyuhong]

Thanks, @pcmoritz for including the scope enum change in Arrow. It looks like this commit does not contains my latest PR of adding deleting a list of objects. That's fine I will do the change later.

[pcmoritz]

@guoyuhong Yeah, I started the PR before yours got merged; I'll probably update it before this PR gets merged, but before that happens, I first need to fix the non-determinism problem.

[robertnishihara]

Probably won't change anything, but we should test to see if this has any performance implications.

[AmplabJenkins]

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/6484/
Test FAILed.

[AmplabJenkins]

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/6490/
Test FAILed.

[AmplabJenkins]

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/6503/
Test FAILed.

[AmplabJenkins]

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/6505/
Test FAILed.

[guoyuhong]

If someone want to update the arrow package, will this patch break the building process?

[guoyuhong]

FYI. My PR which Robert mentioned is used to solve the parquet building error and make it friendly when we update the Arrow package.

[pcmoritz]

@guoyuhong The patch is just a stop gap solution we are using until apache/arrow#2216 is fixed (hopefully soon, but it's hard to tell and we would like to get a release candidate for 0.5 really soon so we can start the testing), then this patch will go away again. It should be independent of your PR.

[AmplabJenkins]

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/6509/
Test FAILed.

[pcmoritz]

retest this please

[pcmoritz]

Performance implications:

Without this PR:

 In [13]: d = 2000 * [{"hello": 100*["world"], "this": 100*[4234], "array": np.zeros(100)}]

In [14]: %timeit ray.put(d)
10 loops, best of 3: 96.8 ms per loop

With this PR:

In [7]: %timeit ray.put(d)
10 loops, best of 3: 101 ms per loop

[AmplabJenkins]

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/6515/
Test PASSed.

[robertnishihara]

Test failures seem unrelated (due to a Travis rebasing issue I think).