[Java] Simplify Java worker configuration #2938

jovany-wang · 2018-09-25T02:11:44Z

What do these changes do?

Previously, Java worker configuration is complicated, because it requires setting environment variables as well as command-line arguments.

This PR aims to simplify Java worker's configuration.

Configuration management is now migrated to lightbend config, thus doesn't require setting environment variables.
Many unused config items are removed.
Provide a simple example.conf file, so users can get started quickly.
All possible options and their default values are declared and documented in ray.default.conf file.

This PR also simplifies and refines the following code:

The process of Ray.init().
RunManager.
WorkerContext.

How to use this configuration?

Copy example.conf into your classpath and rename it to ray.conf.
Modify/add your configuration items. The all items are declared in ray.default.conf.
You can also set the items in java system prosperities.

Note: configuration is read in this priority:
System properties > ray.conf > ray.default.conf

Related issue number

N/A

jovany-wang · 2018-09-25T02:13:13Z

@chuxi cc

AmplabJenkins · 2018-09-25T03:36:44Z

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/8338/
Test PASSed.

robertnishihara · 2018-09-25T03:49:58Z

java/runtime/src/main/resources/ray.default.conf

+    // Redis server, Raylet and object store.
+    address: ""
+    // If `redis.server` isn't provided, which port we should use to start redis server.
+    head-port: 32100


maybe 6379 since that is standard?

robertnishihara · 2018-09-25T03:52:29Z

java/example.conf

+
+  // Path of your Ray home dir,
+  // can be either an absolute path, or a relative path to current work dir.
+  home = "/path/to/your/ray/home"


Is this the ray/ directory where Ray is installed? E.g., if I clone the Ray repository to /home/ubuntu/ray and then build it from source in place, then this should be /home/ubuntu/ray?

"current working directory" sounds like something else, e.g., that sounds more like where my application code lives. So maybe this comment can be clarified?

Is this the ray/ directory where Ray is installed?

yes.

"current working directory" sounds like something else, e.g., that sounds more like where my application code lives. So maybe this comment can be clarified?

The motivation of allowing 'relative path to current work dir' is the following: when we run java tests under ray/java/test folder, ray home will be ../.. of current work dir, see TestListener.java. If we only allow absolute path, everyone needs to change it before running tests.

I see, then can we rephrase the comment as

// This is the path to the directory where Ray is installed, e.g., // something like /home/ubuntu/ray. This can be an absolute path // or a relative path from the current working directory.

To clarify, is it a relative path from the directory that the user is in? Or it a relative path from where the application lives?

It's the relative path from where the program is started, i.e., result of pwd.

@jovan-wong could you update the comment according to @robertnishihara's suggestion?

robertnishihara · 2018-09-25T03:52:58Z

java/example.conf

+  // Worker's mode, available options are:
+  // `WORKER`: indicates that this worker is a normal worker.
+  // `DRIVER`: indicates that this worker is a driver.
+  worker.mode = DRIVER


Under what circumstances should this be set to WORKER or DRIVER?

WORKER should only be set in worker process (the process forked from worker_pool.cc). Actually I think this shouldn't be exposed to user.

I'm a bit confused about this. To run an application, does the user need to make one conf file for the cluster, one for the application, or one for every worker?

Yep. I think we should add an API to specify the worker mode:
For a normal worker, we can invoke this:

Ray.init(WorkerMode.WORKER);

And call ray.init() without argument for a driver.

It's unclear to me what Ray.init(WorkerMode.WORKER); means. Under what circumstances would someone do that? Just for testing/debugging?

User should call Ray.init() instead of Ray.init(WorkerMode.WORKER).
Because Ray.init() will specify the worker mode as DRIVER.
Ray.init(WorkerMode.WORKER) just for the DefaultWorker.java.

And then, we can remove the worker mode item from config file.

I'm a bit confused about this. To run an application, does the user need to make one conf file for the cluster, one for the application, or one for every worker?

there should be only one file for the application. And worker processes should use the same file, except overriding ray.worker.mode=WORKER (see DefaultWorker.java).

Thus, users always call Ray.init(), and don't need to worry about which mode they should set.

robertnishihara · 2018-09-25T03:54:59Z

java/example.conf

+  // `SINGLE_PROCESS`: Ray will be running in a single Java process.
+  // `SINGLE_BOX`: All ray processes will be running on one node.
+  // `CLUSTER`: Ray will be running in a cluster.
+  run-mode = SINGLE_BOX


At a high level, is the conf file for the "cluster" or do you need one conf file for each application?

Why do you need to specify SINGLE_BOX versus CLUSTER?

this file defines the default values, each application should provide their own file to fit their scenario.

this is due to a legacy reason. Because SINGLE_BOX and CLUSTER use different mechanism to manage functions. Once we finish [java] put function meta in task spec and load functions with function meta #2881, SINGLE_BOX can be removed.

AmplabJenkins · 2018-09-25T05:12:38Z

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/8339/
Test PASSed.

AmplabJenkins · 2018-09-25T06:48:19Z

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/8340/
Test FAILed.

AmplabJenkins · 2018-09-25T08:08:11Z

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/8342/
Test FAILed.

raulchen · 2018-09-25T12:44:14Z

java/example.conf

+# For config file format, see 'https://github.com/lightbend/config/blob/master/HOCON.md'.
+
+ray {
+  // This is the path to the directory where Ray in installed, e.g.,


typo: in -> is

AmplabJenkins · 2018-09-25T19:59:55Z

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/8347/
Test PASSed.

AmplabJenkins · 2018-09-25T20:10:00Z

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/8348/
Test PASSed.

AmplabJenkins · 2018-09-26T03:01:47Z

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/8351/
Test PASSed.

raulchen · 2018-09-26T03:02:39Z

java/test/src/main/java/org/ray/api/test/RayConfigTest.java

@@ -13,7 +13,6 @@ public void testCreateRayConfig() {
    System.setProperty("ray.home", "/path/to/ray");
    RayConfig rayConfig = RayConfig.create();

-    Assert.assertEquals(rayConfig.rayHome, "/path/to/ray");


why remove this?

This assert can't pass in my machine.
It always be the path setted here.

Maybe we can remove this lineSystem.setProperty("ray.home", "/path/to/ray");,
and rephrase the assert toassert(rayConfig.rayHome, "../..");

raulchen · 2018-09-26T03:02:45Z

java/test/src/main/java/org/ray/api/test/ResourcesManagementTest.java

@@ -37,7 +37,7 @@ public Integer echo(Integer number) {
    }
  }

-  @RayRemote(resources = {@ResourceItem(name = "CPU", value = 8),
+  @RayRemote(resources = {@ResourceItem(name = "CPU", value = 1000),


why change this?

In different machines, the number of CPUs is not determined, We should make sure that the machine doesn't have enough CPUs.

then, we should set a FIXED number for this test.

AmplabJenkins · 2018-09-26T04:46:36Z

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/8354/
Test PASSed.

robertnishihara · 2018-09-26T06:06:23Z

I think I introduced conflicts by merging #2881.

AmplabJenkins · 2018-09-26T06:36:31Z

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/8357/
Test PASSed.

Remove useless fields in config file. Remove useless fields in PathConfig. Remove all fields and methods in PathConfig. Remove PathConfig. Remove all fields of config file. Remove Parameters. Add overwrite field. Pass iniconfig. Make DefaultWorker using RayAPI to init itself. Remove useless code. Clean code. Remove TODO. Change some thing Add RayConfig. Read config files. Change a little. Add ray.home directory. Update WorkerMode. Update RunMode. Update node ip. Update redis address configuration item. Update object store name configuration item. Update object store name index configuration item. Update some configuration items. Remove useless params. Remove RayParameters and ConfigReader classes. Remove Aconfig and refactor init Rename a func. Remove useless files. Clean code. Clean code. [VERY IMPORTANT] Change module and package dependencies. In this commit, I move some dependencies of RayConfig into api module from runtime module. Remove RayInitConfig. Finish TODOs and address comments. Fix lint. Fix lint. Move DefaultRayRuntimeFactory into runtime module. Refactor WorkerContext In this commit, I removed all the static variables and methods and make it non-static. Fix lint. Address comments. refine RunManager fix fix logger fix test refine update doc and set library at runtime Fix lint:Add a new line. Update README.rst Update the format. Fix: code style. Address comments. Fix Worker Mode. Update ray.conf Fix typo. Fix lint. Update resource management test.

AmplabJenkins · 2018-09-26T10:25:11Z

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/8363/
Test FAILed.

raulchen · 2018-09-26T10:28:55Z

Jenkins, retest this please.

AmplabJenkins · 2018-09-26T10:40:09Z

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/8364/
Test FAILed.

AmplabJenkins · 2018-09-26T10:40:41Z

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/8362/
Test FAILed.

raulchen · 2018-09-26T10:57:17Z

jenkins, retest this please

AmplabJenkins · 2018-09-26T11:08:59Z

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/8366/
Test PASSed.

AmplabJenkins · 2018-09-26T11:39:07Z

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/8368/
Test FAILed.

AmplabJenkins · 2018-09-26T11:44:16Z

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/8367/
Test PASSed.

AmplabJenkins · 2018-09-26T11:44:32Z

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/8369/
Test PASSed.

AmplabJenkins · 2018-09-26T12:13:44Z

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/8372/
Test PASSed.

AmplabJenkins · 2018-09-26T12:25:14Z

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/8370/
Test PASSed.

jovany-wang requested a review from raulchen September 25, 2018 02:11

robertnishihara reviewed Sep 25, 2018

View reviewed changes

raulchen changed the title ~~[Java] Refactor RayConfig and improve Java worker.~~ [Java] Simplify configuration usage of Java worker Sep 25, 2018

raulchen changed the title ~~[Java] Simplify configuration usage of Java worker~~ [Java] Simplify Java worker configuration Sep 25, 2018

raulchen reviewed Sep 25, 2018

View reviewed changes

raulchen reviewed Sep 26, 2018

View reviewed changes

raulchen mentioned this pull request Sep 26, 2018

[Java] RayCli submit function fixing #2956

Closed

raulchen mentioned this pull request Sep 26, 2018

[Java] support ray java assembly package #2866

Closed

jovany-wang force-pushed the simplify_config branch from 191cda4 to 241cf83 Compare September 26, 2018 09:05

jovany-wang and others added 7 commits September 26, 2018 18:14

Fix code style.

87214d3

Update RayConfigTest.

c9f62b3

Remove runtime configs from config file

37205ab

remove SINGLE_BOX mode

cc52c08

Fix checkstyle failure.

53079eb

set workerContext.driverId

c5bb80f

raulchen added 2 commits September 26, 2018 18:15

update doc

743a5a7

fix

e06755f

raulchen force-pushed the simplify_config branch from bd4645a to e06755f Compare September 26, 2018 10:16

fix format

ca3b719

raulchen approved these changes Sep 26, 2018

View reviewed changes

raulchen merged commit 8e8e123 into ray-project:master Sep 26, 2018

raulchen deleted the simplify_config branch September 26, 2018 12:49

[Java] Simplify Java worker configuration #2938

[Java] Simplify Java worker configuration #2938

Conversation

jovany-wang commented Sep 25, 2018 • edited by raulchen Loading

What do these changes do?

How to use this configuration?

Related issue number

jovany-wang commented Sep 25, 2018

AmplabJenkins commented Sep 25, 2018

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

raulchen Sep 25, 2018 • edited Loading

Choose a reason for hiding this comment

AmplabJenkins commented Sep 25, 2018

AmplabJenkins commented Sep 25, 2018

AmplabJenkins commented Sep 25, 2018

Choose a reason for hiding this comment

AmplabJenkins commented Sep 25, 2018

AmplabJenkins commented Sep 25, 2018

AmplabJenkins commented Sep 26, 2018

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

AmplabJenkins commented Sep 26, 2018

robertnishihara commented Sep 26, 2018

AmplabJenkins commented Sep 26, 2018

AmplabJenkins commented Sep 26, 2018

raulchen commented Sep 26, 2018

AmplabJenkins commented Sep 26, 2018

AmplabJenkins commented Sep 26, 2018

raulchen commented Sep 26, 2018

AmplabJenkins commented Sep 26, 2018

AmplabJenkins commented Sep 26, 2018

AmplabJenkins commented Sep 26, 2018

AmplabJenkins commented Sep 26, 2018

AmplabJenkins commented Sep 26, 2018

AmplabJenkins commented Sep 26, 2018

jovany-wang commented Sep 25, 2018 •

edited by raulchen

Loading

raulchen Sep 25, 2018 •

edited

Loading