Enable to specify driver id by user. #3084

jovany-wang · 2018-10-18T07:00:07Z

What do these changes do?

It's very useful for us to specify driver id to a driver.
In Java, we could specify it by ray.conf, and in Python, we should do it by def init():

ray.init(driver=b"01234567890123456789")

If we don't specify one, it will generate a random for us.

Related issue number

N/A

robertnishihara · 2018-10-18T07:04:21Z

python/ray/worker.py

@@ -1664,6 +1667,7 @@ def init(redis_address=None,
         plasma_directory=None,
         huge_pages=False,
         include_webui=True,
+         worker_id=None,


If this is the driver_id then it needs to be called driver_id.

robertnishihara · 2018-10-18T07:05:51Z

python/ray/worker.py

@@ -2017,7 +2026,10 @@ def connect(info,
    assert not worker.connected, error_message
    assert worker.cached_functions_to_run is not None, error_message
    # Initialize some fields.
-    worker.worker_id = random_string()
+    if worker_id is None:


We should do some type checking. I think we want to assert that it is a Ray ObjectID or that it is 20 bytes or something like that. All of that is a bit ugly unfortunately.

robertnishihara · 2018-10-18T07:07:35Z

I have mixed feelings about this. I think we want to restrict driver_id's to have type ray.DriverID, but we don't have that, so it would need to be ray.ObjectID, which feels weird because it's a driver, not an object. You're code example seems to use a short string, which doesn't make sense to me. Doesn't it need to be 20 bytes?

AmplabJenkins · 2018-10-18T09:03:29Z

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/8729/
Test PASSed.

AmplabJenkins · 2018-10-18T11:38:56Z

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/8732/
Test PASSed.

jovany-wang · 2018-10-18T15:53:11Z

@robertnishihara Thanks a lot for your comments about this.

I have addressed some comments:
adding type checking and renaming worker_id to driver_id.

It's too awkward for me to do type checking for these codes. As @robertnishihara mentioned, which type should driver_id be? Maybe we should add an UniqueID type.

Another question I found is worker.worker_id's type. I guess we assume that it's a string or bytes. One problem that comes with this is :

    # Initialize some fields.
    if mode is WORKER_MODE:
        worker.worker_id = random_string()
    else:
        # This is the code path of driver mode.
        if driver_id is None:
            driver_id = ray.ObjectID(random_string())

        elif isinstance(driver_id, (bytes, str)):
            if len(driver_id) is not ray_constants.ID_SIZE:
                raise Exception("The length of given driver id str must be 20.")
            driver_id = ray.ObjectID(driver_id)

        elif not isinstance(driver_id, ray.ObjectID):
            raise Exception("The type of given driver id must be str or ray.ObjectID.")

        worker.worker_id = driver_id.id()

We should assume driver_id is a ray.ObjectID because we should cast from a str or a ray.ObjectID, And the next we should cast driver_id to str(or bytes) to match worker.worker_id.

I think the better way to solve it is to use worker_id as ray.ObjectID instead of str(or bytes). And we needn't worry about the length of worker_id any more, right?

jovany-wang · 2018-10-18T16:01:31Z

@guoyuhong cc

AmplabJenkins · 2018-10-18T16:24:18Z

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/8734/
Test PASSed.

AmplabJenkins · 2018-10-19T04:20:48Z

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/8746/
Test PASSed.

robertnishihara · 2018-10-19T05:49:04Z

@guoyuhong @jovan-wong what is the use case for specifying the driver ID?

guoyuhong · 2018-10-19T06:23:21Z

@robertnishihara We will enable cross-language calling which makes streaming work smoothly. Before Java Driver calling a Python worker, the python code should be deployed to a path or vise versa. Knowing the driver Id before hand will enable use to deploy the python/java code to a directory related to the driver id.

raulchen · 2018-10-22T07:17:22Z

Another purpose is that we want to have an external 'job manager' that runs all drivers with pre-defined driver ids, so it can manage the drivers by the ids.

@robertnishihara do you have any other questions? This PR looks good to me.

AmplabJenkins · 2018-10-22T13:57:59Z

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/8795/
Test FAILed.

AmplabJenkins · 2018-10-22T14:41:00Z

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/8797/
Test FAILed.

robertnishihara · 2018-10-29T01:50:11Z

python/ray/worker.py

+        if driver_id is None:
+            driver_id = ray.ObjectID(random_string())
+
+        elif isinstance(driver_id, (bytes, str)):


Let's check that it is bytes for Python 2 and str for Python 3.

actually, for now, let's require it to be an ObjectID. I think the more restrictive we make it the better since it makes it easier to change later.

robertnishihara · 2018-10-29T01:51:33Z

python/ray/worker.py

@@ -1731,6 +1736,7 @@ def init(redis_address=None,
            Store with hugetlbfs support. Requires plasma_directory.
        include_webui: Boolean flag indicating whether to start the web
            UI, which is a Jupyter notebook.
+        driver_id: The id of driver, and it will be ignored in non-driver mode.


id -> ID

remove the stuff after the ,, there is no such thing as "non-driver" mode.

at least from the perspective of the user and ray.init, there is no such thing as "non-driver" mode.

robertnishihara

Thanks! I left some comments.

Add type checking and address comments. Fix small thing. Add ut. Fix lint Remove useless code. Address comments.

jovany-wang · 2018-11-02T09:18:02Z

@robertnishihara Tks 4 ur comments.
I have addressed them, and could u plz c it again?

AmplabJenkins · 2018-11-02T10:29:30Z

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/9000/
Test FAILed.

AmplabJenkins · 2018-11-02T10:40:33Z

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/9001/
Test PASSed.

AmplabJenkins · 2018-11-02T10:41:34Z

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/9002/
Test PASSed.

robertnishihara · 2018-11-02T22:28:15Z

@jovan-wong thanks, I fixed some linting errors. It can be merged once tests pass.

AmplabJenkins · 2018-11-03T00:00:05Z

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/9014/
Test PASSed.

robertnishihara reviewed Oct 18, 2018

View reviewed changes

raulchen approved these changes Oct 22, 2018

View reviewed changes

robertnishihara reviewed Oct 29, 2018

View reviewed changes

robertnishihara requested changes Oct 29, 2018

View reviewed changes

jovany-wang force-pushed the enable-driver-id branch from 2be4a4b to a484a9c Compare November 2, 2018 09:07

Enable specifing driver or worker id.

202d6bd

Add type checking and address comments. Fix small thing. Add ut. Fix lint Remove useless code. Address comments.

jovany-wang force-pushed the enable-driver-id branch from a484a9c to 202d6bd Compare November 2, 2018 09:15

robertnishihara approved these changes Nov 2, 2018

View reviewed changes

Update runtest.py

9eaed2a

robertnishihara merged commit ca7d4c2 into ray-project:master Nov 3, 2018

jovany-wang deleted the enable-driver-id branch November 13, 2018 10:46

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Enable to specify driver id by user. #3084

Enable to specify driver id by user. #3084

jovany-wang commented Oct 18, 2018 •

edited

Loading

robertnishihara Oct 18, 2018 •

edited

Loading

robertnishihara Oct 18, 2018

robertnishihara commented Oct 18, 2018

AmplabJenkins commented Oct 18, 2018

AmplabJenkins commented Oct 18, 2018

jovany-wang commented Oct 18, 2018 •

edited

Loading

jovany-wang commented Oct 18, 2018

AmplabJenkins commented Oct 18, 2018

AmplabJenkins commented Oct 19, 2018

robertnishihara commented Oct 19, 2018

guoyuhong commented Oct 19, 2018

raulchen commented Oct 22, 2018

AmplabJenkins commented Oct 22, 2018

AmplabJenkins commented Oct 22, 2018

robertnishihara Oct 29, 2018

robertnishihara Oct 29, 2018

robertnishihara Oct 29, 2018

robertnishihara Oct 29, 2018

robertnishihara left a comment

jovany-wang commented Nov 2, 2018

AmplabJenkins commented Nov 2, 2018

AmplabJenkins commented Nov 2, 2018

AmplabJenkins commented Nov 2, 2018

robertnishihara commented Nov 2, 2018

AmplabJenkins commented Nov 3, 2018

Enable to specify driver id by user. #3084

Enable to specify driver id by user. #3084

Conversation

jovany-wang commented Oct 18, 2018 • edited Loading

What do these changes do?

Related issue number

robertnishihara Oct 18, 2018 • edited Loading

Choose a reason for hiding this comment

robertnishihara Oct 18, 2018

Choose a reason for hiding this comment

robertnishihara commented Oct 18, 2018

AmplabJenkins commented Oct 18, 2018

AmplabJenkins commented Oct 18, 2018

jovany-wang commented Oct 18, 2018 • edited Loading

jovany-wang commented Oct 18, 2018

AmplabJenkins commented Oct 18, 2018

AmplabJenkins commented Oct 19, 2018

robertnishihara commented Oct 19, 2018

guoyuhong commented Oct 19, 2018

raulchen commented Oct 22, 2018

AmplabJenkins commented Oct 22, 2018

AmplabJenkins commented Oct 22, 2018

robertnishihara Oct 29, 2018

Choose a reason for hiding this comment

robertnishihara Oct 29, 2018

Choose a reason for hiding this comment

robertnishihara Oct 29, 2018

Choose a reason for hiding this comment

robertnishihara Oct 29, 2018

Choose a reason for hiding this comment

robertnishihara left a comment

Choose a reason for hiding this comment

jovany-wang commented Nov 2, 2018

AmplabJenkins commented Nov 2, 2018

AmplabJenkins commented Nov 2, 2018

AmplabJenkins commented Nov 2, 2018

robertnishihara commented Nov 2, 2018

AmplabJenkins commented Nov 3, 2018

jovany-wang commented Oct 18, 2018 •

edited

Loading

robertnishihara Oct 18, 2018 •

edited

Loading

jovany-wang commented Oct 18, 2018 •

edited

Loading