Use RAY_ADDRESS to connect to an existing Ray cluster if present

[pcmoritz]

Why are these changes needed?

We would like to allow people to run existing Ray applications on a cluster without changing their code. This means if they use a driver that is initialized with ray.init(), there should be a way to make it connect to an existing cluster instead of starting a new one. This can make it more robust to execute Ray applications, i.e. it can force connecting to an existing cluster and avoid the possible problem where somebody runs an application that starts a new local Ray instead of using the existing cluster.

This PR makes that possible by introducing an environment variable RAY_ADDRESS. If it is defined while the driver is run, it will be treated as the address parameter of ray.init, making it connect to the existing cluster.

Q & A:

• Should we print a warning if the RAY_ADDRESS variable is set and we connect to an existing cluster?

In general it is not common practice to print a warning if an environment variable is picked up, but in this case there could be a case for it, especially since warnings can be turned off selectively. On the other hand, if RAY_ADDRESS is set, using the existing cluster should be the expected behaviour.

• Should we have an option to force a local cluster even if RAY_ADDRESS is defined?

This seems like a rare use case and it is already supported by doing

del os.environ["RAY_ADDRESS"]
ray.init()

So I'd avoid complicating the API for this.

Related issue number

Checks

• I've run scripts/format.sh to lint the changes in this PR.
• I've included any doc changes needed for https://ray.readthedocs.io/en/latest/.
• I've made sure the tests are passing. Note that there might be a few flaky tests, see the recent failure rates at https://ray-travis-tracker.herokuapp.com/.
• Testing Strategy
  • Unit tests
  • Release tests
  • This PR is not tested (please justify below)

[AmplabJenkins]

Can one of the admins verify this patch?

[AmplabJenkins]

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/24548/
Test PASSed.

[AmplabJenkins]

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/25206/
Test PASSed.

[AmplabJenkins]

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/25204/
Test PASSed.

[AmplabJenkins]

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/25207/
Test PASSed.

[AmplabJenkins]

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/25208/
Test PASSed.

[AmplabJenkins]

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/25210/
Test FAILed.

[edoakes]

LGTM! I don't think we need to print a warning if connecting. It seems uncommon for a user to want to start a local cluster when there's one already running on the machine anyways. If there isn't and RAY_ADDRESS is picked up, it will fail anyways.