enable starting remote ray nodes with raylet

[atumanov]

What do these changes do?

We're passing use_raylet for non-head ray nodes, to enable non-head nodes to be started with raylet instead of the local_scheduler. Currently, remote nodes start with local_scheduler even when --use-raylet is specified on the command line. This may resolve #1870.

Related issue number

#1870

[pcmoritz]

Great catch!

[atumanov]

a simple test with two actors distributes the actors across the two raylet nodes. To reproduce:

from __future__ import absolute_import
from __future__ import division
from __future__ import print_function

import ray
import sys

if len(sys.argv) < 2:
  print("USAGE: %s redis_ip:redis_port")
  sys.exit(1)

redis_addr = sys.argv[1]

@ray.remote(resources={"Actor1": 1})
class Worker1(object):

    def __init__(self):
        pass

    def get_node_ip(self):
        return ray.services.get_node_ip_address()

@ray.remote(resources={"Actor2" : 1})
class Worker2(object):

    def __init__(self):
        pass

    def get_node_ip(self):
        return ray.services.get_node_ip_address()

ray.init(redis_address=redis_addr, use_raylet=True)
ws1 = Worker1.remote()
ws2 = Worker2.remote()

print("XXX", [ray.get(ws1.get_node_ip.remote()), ray.get(ws2.get_node_ip.remote())])

Output:

XXX ['172.31.9.114', '172.31.10.237']

[robertnishihara]

Good catch!

[AmplabJenkins]

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/4816/
Test PASSed.