[autoscaler] bash 4.4 does not support set -i

[atypic]

System information

• OS Platform and Distribution (e.g., Linux Ubuntu 16.04):
  Ubuntu 17.10

• Ray installed from (source or binary):
  Source

• Ray version:
  commit b8811cb

• Python version:
  3.6

• Exact command to reproduce:
  ray create_or_update example.yaml

Describe the problem

As of bash 4.4, set -i is no longer accepted to create an interactive shell. Consider -t in stead.

Source code / logs

[atypic]

Tried using -t, this creates problems down the line. Will try to come up with a better suggestion...

[atypic]

OTOH -- why try to force interactive shells anyway?

[robertnishihara]

Thanks @atypic, I believe the motivation was so that the sequence of commands, e.g.,

ray/python/ray/autoscaler/aws/development-example.yaml

Lines 69 to 80 in 215d526

	setup_commands:
	# Install basics.
	- sudo apt-get update
	- sudo apt-get install -y cmake pkg-config build-essential autoconf curl libtool unzip python
	# Install Anaconda.
	- wget https://repo.continuum.io/archive/Anaconda3-5.0.1-Linux-x86_64.sh || true
	- bash Anaconda3-5.0.1-Linux-x86_64.sh -b -p $HOME/anaconda3 || true
	- echo 'export PATH="$HOME/anaconda3/bin:$PATH"' >> ~/.bashrc
	# Build Ray.
	- git clone https://github.com/ray-project/ray || true
	- pip install -U cloudpickle boto3==1.4.8
	- cd ray/python; python setup.py develop

would share "state" with each other. E.g., modifying the PATH in one command carries over to the next.

[ericl]

@robertnishihara the commands are actually still separate sessions. The original motivation was so that the setup commands can run with a similar PATH as an interactive shell (otherwise, you'd have to prepend all your commands with things like PATH=~/anaconda3/bin:$PATH).

You might think that just auto-prepending source ~/.bashrc to each command would be sufficient, but it turns out many bashrc scripts early exit if the session is non-interactive. Hence the set -i.

One solution might just be to remove the set -i, this is not as nice from the user perspective but avoids weird pitfalls.

[ericl]

This stackoverflow post has a better explanation: https://stackoverflow.com/questions/940533/how-do-i-set-path-such-that-ssh-userhost-command-works

[atypic]

I tried to remove set -i and modify the yaml accordingly, however something odd happens to the startup of the redis server, making me unable to connect to the socket. The server is running and all seems fine, but ray.init() fails ("Unable to connect to socket /tmp/"...). I do something similar to

source /home/ubuntu/anaconda3/bin/activate MyEnvironment; ray start --head <...>
in the head_start_ray_commands. I had to go back to bash 4.3 to get set -i.

ray start perhaps depends on something in Env/.bashrc? I have yet to investigate :-)

Anyway...
Somehow depending on the behavior of bash seems wonky, as people like to use a variety of shells even on AWS.

Another solution could perhaps be to have a "PATH" section in the yaml, and use .ssh/environment.

Node provisioning is always such a mess :(