Skip to content

New way to install flow #62

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 20 commits into from
Sep 3, 2018
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
90 changes: 90 additions & 0 deletions docker/Dockerfile-RemoteDesktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,90 @@
FROM consol/ubuntu-xfce-vnc

USER 0

ENV LANG=C.UTF-8 LC_ALL=C.UTF-8
ENV PATH /opt/conda/bin:$PATH

RUN apt-get update --fix-missing && apt-get install -y wget bzip2 ca-certificates \
libglib2.0-0 libxext6 libsm6 libxrender1 \
git mercurial subversion

RUN wget --quiet https://repo.anaconda.com/miniconda/Miniconda3-4.5.4-Linux-x86_64.sh -O ~/miniconda.sh && \
/bin/bash ~/miniconda.sh -b -p /opt/conda && \
rm ~/miniconda.sh && \
ln -s /opt/conda/etc/profile.d/conda.sh /etc/profile.d/conda.sh && \
echo ". /opt/conda/etc/profile.d/conda.sh" >> ~/.bashrc && \
echo "conda activate base" >> ~/.bashrc

RUN apt-get install -y curl grep sed dpkg && \
TINI_VERSION=`curl https://github.com/krallin/tini/releases/latest | grep -o "/v.*\"" | sed 's:^..\(.*\).$:\1:'` && \
curl -L "https://github.com/krallin/tini/releases/download/v${TINI_VERSION}/tini_${TINI_VERSION}.deb" > tini.deb && \
dpkg -i tini.deb && \
rm tini.deb && \
apt-get clean


RUN apt-get update && apt-get upgrade -y
RUN apt-get install -y cmake swig libgtest-dev python-pygame python-scipy autoconf libtool pkg-config libgdal-dev libxerces-c-dev && \
apt-get install -y libproj-dev libfox-1.6-dev libxml2-dev libxslt1-dev build-essential curl unzip flex bison python python-dev python3-dev
RUN pip install cmake cython



RUN cd ~ && \
git clone --recursive https://github.com/eclipse/sumo.git && \
cd sumo && \
git checkout cbe5b73d781376c939b07c6127803a804c803ed7 && \
mkdir build/cmake-build

RUN cd ~/sumo && cd build/cmake-build && \
cmake ../.. && \
make && \
echo 'export SUMO_HOME="$HOME/sumo"' >> ~/.bashrc && \
echo 'export PATH="$HOME/sumo/bin:$PATH"' >> ~/.bashrc && \
echo 'export PYTHONPATH="$HOME/sumo/tools:$PYTHONPATH"' >> ~/.bashrc





RUN cd ~ && \
git clone https://github.com/flow-project/flow.git && \
cd flow && \
conda env create -f environment.yml

RUN cd flow && python setup.py develop && \
echo 'export PYTHONPATH="$HOME/flow:$PYTHONPATH"' >> ~/.bashrc

RUN cd ~ && \
git clone https://github.com/cathywu/rllab-multiagent.git && \
cd rllab-multiagent && \
conda env create -f environment.yml && \
python setup.py develop && \
echo 'export PYTHONPATH="$HOME/rllab-multiagent:$PYTHONPATH"' >> ~/.bashrc

RUN echo "source activate flow" >> /root/.bashrc
RUN echo "source activate flow"
ENV BASH_ENV /root/.bashrc




## RAY
RUN apt-get update

RUN apt-get install -y libboost-dev libboost-test-dev libboost-program-options-dev libboost-filesystem-dev \
libboost-thread-dev libevent-dev automake libtool flex \
bison pkg-config g++ libssl-dev
RUN git clone https://github.com/apache/thrift && cd thrift && ./bootstrap.sh && ./configure && make && make install

#### RAY
RUN cd ~ conda install libgcc jupyter
RUN apt-get update
RUN apt-get install -y cmake pkg-config build-essential autoconf curl libtool libboost-dev libboost-filesystem-dev libboost-system-dev unzip bison openssl
RUN git clone https://github.com/eugenevinitsky/ray
RUN cd ray/python && python setup.py install
RUN echo 'export PYTHONPATH="$HOME/flow:$PYTHONPATH"' >> ~/.bashrc
RUN pip install lz4 gym==0.10.5

USER 0
66 changes: 53 additions & 13 deletions docs/source/flow_setup.rst
Original file line number Diff line number Diff line change
@@ -1,4 +1,7 @@
Setup Instructions
.. contents:: Table of contents


1. Local Installation
******************

To get Flow running, you need three things: Flow,
Expand All @@ -11,7 +14,7 @@ module bugs from Python. Just install the missing module using
your OS-specific package manager / installation tool. Follow the
shell commands below to get started.

Dependencies
1. Dependencies
============
We begin by installing dependencies needed by the four repositories mentioned
above. **It is highly recommended that users install**
Expand All @@ -36,7 +39,7 @@ install it):
# sumo dependencies
brew install Caskroom/cask/xquartz autoconf automake pkg-config libtool gdal proj xerces-c fox

sumo
2. Sumo
====
Next, we install SUMO, an open source traffic microsimulator which will be used
the update the states of vehicles, traffic lights, and other RL and
Expand Down Expand Up @@ -81,11 +84,10 @@ Finally, test your sumo install and version by running the following commands
which sumo
sumo --version
sumo-gui

If you are a mac user and the above command gives you the error FXApp:openDisplay: unable to open display :0.0 make sure to open the application XQuartz.

If you are a mac user and the above command gives you the error ``FXApp:openDisplay: unable to open display :0.0``
make sure to open the application XQuartz.

Flow
3. Flow
====
Once sumo and the various dependencies are in place, we are ready to install a
functional version of Flow. With this, we can begin to simulate traffic within
Expand All @@ -112,7 +114,7 @@ For mac run
echo 'export PYTHONPATH="$HOME/flow:$PYTHONPATH"' >> ~/.bash_profile
source ~/.bash_profile

Testing the Installation
4. Testing the Installation
========================

Once the above modules have been successfully installed, we can test the
Expand All @@ -136,7 +138,7 @@ Optionally, run the unit tests:
Congratulations, you now have successfully set up Flow!


rllab-multiagent (optional)
5. Rllab-multiagent (optional)
===========================
Flow has been tested on a variety of RL libraries, the installation of which is
optional but may be of use when trying to execute some of the examples files
Expand All @@ -159,7 +161,7 @@ For mac run
echo 'export PYTHONPATH="$HOME/rllab-multiagent:$PYTHONPATH"' >> ~/.bash_profile
source ~/.bash_profile

Ray/RLlib (optional)
6. Ray/RLlib (optional)
====================
RLlib is another RL library that has been extensively tested on the Flow
repository.
Expand All @@ -177,7 +179,7 @@ required libraries as specified at
<http://ray.readthedocs.io/en/latest/installation.html> and
then follow the setup instructions.

Getting started (rllab-multiagent)
7. Getting started (rllab-multiagent)
==================================

To run any of the RL examples, make sure to run
Expand All @@ -195,7 +197,7 @@ If it does not fail, this means that you have Flow properly configured with
rllab-multiagent.


Getting started (Ray/RLlib)
8. Getting started (Ray/RLlib)
===========================

See `getting started with RLlib <http://ray.readthedocs.io/en/latest/rllib.html#getting-started>`_ for sample commands.
Expand Down Expand Up @@ -228,10 +230,48 @@ jobs from there.
ray teardown scripts/ray_autoscale.yaml


Custom configuration
9. Custom configuration
====================

You may define user-specific config parameters as follows
::

cp flow/core/config.template.py flow/core/config.py # Create template for users using pycharm


2. Remote desktop using Docker for simple utilisation of flow (recommended for testing purpose)
******************

1. Installation
====================

Installation of a remote desktop and docker to get access to flow quickly

First install docker on https://www.docker.com/

In terminal
::
1° docker pull lucasfischerberkeley/flowdesktop
2° docker run -d -p 5901:5901 -p 6901:6901 -p 8888:8888 lucasfischerberkeley/flowdesktop

Go into your browser ( Firefox, Chrome, Safari)
::
1° Go to http://localhost:6901/?password=vncpassword
2° Go to Applications and open Terminal Emulator
3° For sumo: Write python flow/examples/sumo/sugiyama.py and run it
4° For rllib : Write python flow/examples/rllib/stabilizing_the_ring.py and run it
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

so rllib is not installed in a virtual env but rllab is?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@eugenevinitsky. Rllib is installed in the root environment and works.
For rllab, I need to enter another environment ( flow-rllab) but I am working on a way to have everything set up in the same environment

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Okay, should be good to go after this. Can you add the dockerfile that you used to create the docker image? Would be useful to have in the docker folder as a record if we need to remake it!

5° For rllab : source activate flow-rllab and python flow/examples/rllab/figure_eight.py ( first time, run it twice)


2. Notebooks and tutorial
====================

In the docker desktop
::
1° Go into Terminal Emulator
2° Run jupyter notebook --NotebookApp.token=admin --ip 0.0.0.0 --allow-root

Go into your browser ( Firefox, Chrome, Safari)
::
1° go to localhost:8888/tree
2° the password is 'admin' and you can run all your notebook and tutorial