workaround for python3.5 fast numpy serialization

[suquark]

Why are these changes needed?

workaround for python3.5 fast numpy serialization, it is necessary for pickle5 support with python3.5

Related issue number

Checks

• I've run scripts/format.sh to lint the changes in this PR.
• I've included any doc changes needed for https://ray.readthedocs.io/en/latest/.
• I've made sure the tests are passing. Note that there might be a few flaky tests, see the recent failure rates at https://ray-travis-tracker.herokuapp.com/.

[AmplabJenkins]

Can one of the admins verify this patch?

[AmplabJenkins]

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/20308/
Test PASSed.

[AmplabJenkins]

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/20428/
Test FAILed.

[suquark]

It seems that we have pass all related tests, I can start removing arrow serialization now.

[AmplabJenkins]

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/20426/
Test PASSed.

[AmplabJenkins]

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/20482/
Test PASSed.

[suquark]

@pcmoritz I think this PR is ready after the linting. I have removed the pyarrow serialization code path.

[AmplabJenkins]

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/20493/
Test PASSed.

[AmplabJenkins]

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/20494/
Test PASSed.

[edoakes]

@suquark What's the status of this? Would be awesome to get it merged so that we can remove the pyarrow serialization codepath now that we've dropped python 2 support.

[pcmoritz]

The PR looks good to me, two things that need to be done before merging:

• Run the whole test suite on Python 3.5 (Travis is using 3.6)
• Make sure there is no performance regression on numpy arrays (for Python 3.5, 3.6, 3.7, 3.8).

@suquark Can you test this?

[suquark]

@pcmoritz I have performed most tests on python3.5, those tests related to this PR works fine; but some other tests failed in some weird ways (pytest fixtures not found, sigabort received, some python dependencies not found, etc).
I run these tests using pytest under ray/python after compiling the ray from the source code. And installed it using pip install . -e. Is it the correct way for setting up tests locally?

[pcmoritz]

I tested the put performance on Python 3.5, which is fast and also ran the tests on 3.5. There are a couple of failures on 3.5 but they all look unrelated.