[Core] Turning off log redirection to files breaks all Ray service startups. #9541
Description
What is the problem?
Turning off log redirection to files breaks all Ray process startups (redis, GCS server, log monitor, reporter, plasma store, raylet, etc.)
File "/app/descarteslabs/services/workbench/deploy/head-image-dev.binary.runfiles/requirements_py3_pypi__ray_0_9_0_dev0/ray-0.9.0.dev0.data/purelib/ray/scripts/scripts.py", line 470, in start
ray_params, head=True, shutdown_at_exit=block, spawn_reaper=block)
File "/app/descarteslabs/services/workbench/deploy/head-image-dev.binary.runfiles/requirements_py3_pypi__ray_0_9_0_dev0/ray-0.9.0.dev0.data/purelib/ray/node.py", line 194, in __init__
self.start_head_processes()
File "/app/descarteslabs/services/workbench/deploy/head-image-dev.binary.runfiles/requirements_py3_pypi__ray_0_9_0_dev0/ray-0.9.0.dev0.data/purelib/ray/node.py", line 746, in start_head_processes
self.start_redis()
File "/app/descarteslabs/services/workbench/deploy/head-image-dev.binary.runfiles/requirements_py3_pypi__ray_0_9_0_dev0/ray-0.9.0.dev0.data/purelib/ray/node.py", line 506, in start_redis
redis_log_files = [(open_log(redis_out_name),
File "/app/descarteslabs/services/workbench/deploy/head-image-dev.binary.runfiles/requirements_py3_pypi__ray_0_9_0_dev0/ray-0.9.0.dev0.data/purelib/ray/utils.py", line 397, in open_log
return open(path, **kwargs)
TypeError: expected str, bytes or os.PathLike object, not NoneType
Ray version and other system information (Python version, TensorFlow version, OS):
This bug was introduced in this PR, so it shouldn't be affecting any releases.
The issue is that get_log_file_names will return None, None if output redirection is turned off, while all callers of that function now pass the returned log file names directly to open_log, e.g. here.
Reproduction:
Start Ray with the environment variable GLOG_logtostderr set to 1, or with redirect_output or redirect_worker_output set to False. I've confirmed that this bug exists on the latest build off master.
If we cannot run your script, we cannot fix your issue.
- I have verified my script runs in a clean environment and reproduces the issue.
- I have verified the issue also occurs with the latest wheels.