[WIP] Implementing Gcs sharding after putting Gcs data into redis data shard

[heyucongtom]

What do these changes do?

Basically a re-implementation of #2281, with modifications of #2298.
[+] Implement sharding for gcs tables.
[+] Keep ClientTable and ErrorTable managed by the primary_shard. TaskTable is managed by the primary_shard for now, until a good hashing for tasks is implemented.
[+] Move AsyncGcsClient's initialization into Connect function.
[-] Move GetRedisShard and bool sharding from RedisContext's connect into AsyncGcsClient. This may make the interface cleaner.

Please help review the changes and let me know if anything looks problematic.

Related issue number

[AmplabJenkins]

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/6413/
Test PASSed.

[AmplabJenkins]

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/6417/
Test PASSed.

[AmplabJenkins]

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/6419/
Test PASSed.

[AmplabJenkins]

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/6422/
Test PASSed.

[AmplabJenkins]

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/6443/
Test FAILed.

[AmplabJenkins]

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/6445/
Test FAILed.

[AmplabJenkins]

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/6446/
Test FAILed.

[AmplabJenkins]

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/6454/
Test FAILed.

[AmplabJenkins]

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/6455/
Test FAILed.

[AmplabJenkins]

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/6457/
Test PASSed.

[robertnishihara]

@heyucongtom I think some unwanted changes got into this PR (perhaps through some rebasing). Now that #2298 has been merged, I think this PR can be simplified a bit.

[heyucongtom]

Sure. I would take a look. Could you point out some changes that shall be reverted? I do make some changes with regard to #2298. Is that part the problematic one?

…

On Fri, Jul 6, 2018 at 7:30 AM Robert Nishihara ***@***.***> wrote: @heyucongtom <https://github.com/heyucongtom> I think some unwanted changes got into this PR (perhaps through some rebasing). Now that #2298 <#2298> has been merged, I think this PR can be simplified a bit. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#2334 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AGbH8WTSKPt9vrA9tHnlP-fMsTN93mSRks5uDqGAgaJpZM4U-evS> .

[robertnishihara]

E.g., all of the #if !RAY_USE_NEW_GCS changes. The addition of the GetRedisShards function.

[heyucongtom]

Emm these are interesting. Probably I will open up a new PR and work with the current master.

…

On Fri, Jul 6, 2018 at 7:35 AM Robert Nishihara ***@***.***> wrote: E.g., all of the #if !RAY_USE_NEW_GCS changes. The addition of the GetRedisShards function. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#2334 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AGbH8QAb-FK4OFkqCU9T5M8clochInq8ks5uDqLOgaJpZM4U-evS> .