keepaway base implementation

.gitignore

@@ -16,12 +16,13 @@ todos.py
 *.c
 build/
 binary/
-
 test.py
-
 player-*.txt
 player-*.err
-log
 .gitignore
 test-log
 main_tmp.py
+.DS_Store
+keepaway.egg-info/
+logs/
+results/

Dockerfile

@@ -0,0 +1,97 @@
+# Start from Ubuntu 20.04
+FROM ubuntu:20.04
+
+# Avoid prompts from apt during build
+ARG DEBIAN_FRONTEND=noninteractive
+ENV DISPLAY docker.for.mac.host.internal:0
+ENV RCSS_CONF_DIR /home/rcsoccersim/.rcssserver
+ENV LOG_DIR /home/rcsoccersim/logs
+ENV TEAM_DIR /home/rcsoccersim/teams
+USER root
+RUN apt-get update && apt-get upgrade -y
+
+# Install build dependencies for rcssserver and soccerwindow2, including common dependencies
+RUN apt-get install -y build-essential git autoconf libtool qt5-default libqt5opengl5-dev libaudio-dev libxt-dev libxi-dev libxmu-dev libpng-dev libglib2.0-dev libfontconfig1-dev libxrender-dev libxext-dev
+
+RUN apt-get update && apt-get install -y flex automake autoconf libtool flex bison libboost-all-dev
+
+# Clone, build, and install rcssserver
+WORKDIR /root
+RUN git clone https://github.com/rcsoccersim/rcssserver.git
+
+WORKDIR /root/rcssserver
+RUN autoreconf -i && ./configure && make && make install && ldconfig
+
+RUN mkdir -p /root/src
+WORKDIR /root/src
+RUN git clone  https://github.com/helios-base/librcsc.git \
+    && cd librcsc \
+    && ./bootstrap \
+    && ./configure --disable-unit-test \
+    && make \
+    && make install && ldconfig
+
+
+WORKDIR /root/src
+RUN git clone  https://github.com/helios-base/soccerwindow2.git \
+    && cd soccerwindow2 \
+    && autoreconf -i \
+    && ./configure \
+    && make \
+    && make install && ldconfig
+
+
+RUN apt-get update && apt-get install -y wget git bzip2 && \
+    rm -rf /var/lib/apt/lists/*
+
+
+
+RUN wget https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-aarch64.sh -O /miniconda.sh && \
+    bash /miniconda.sh -b -p /miniconda && \
+    rm /miniconda.sh
+
+# Add Conda to PATH
+ENV PATH="/miniconda/bin:${PATH}"
+
+# Create a Conda environment named 'keepaway' with Python 3.11
+RUN conda create -y --name keepaway python=3.11
+
+
+# Cleanup to reduce image size
+RUN apt-get clean && rm -rf /var/lib/apt/lists/* /tmp/* /var/tmp/*
+
+
+RUN useradd -d /home/rcsoccersim -m -s /bin/bash rcsoccersim \
+  && echo "rcsoccersim:rcsoccersim" | chpasswd
+RUN mkdir -p $RCSS_CONF_DIR $TEAM_DIR $LOG_DIR
+RUN chown -R rcsoccersim:rcsoccersim /home/rcsoccersim
+USER rcsoccersim
+
+VOLUME $TEAM_DIR
+VOLUME $LOG_DIR
+
+WORKDIR $TEAM_DIR
+
+## install keepaway 
+
+RUN git clone https://github.com/aabayomi/Pyrus2D.git \ 
+      && cd Pyrus2D
+SHELL ["conda", "run", "-n", "keepaway", "/bin/bash", "-c"] 
+
+WORKDIR $TEAM_DIR/Pyrus2D
+
+COPY entrypoint.sh /usr/local/bin/entrypoint.sh
+
+#COPY requirements.txt /tmp/
+#RUN conda install --no-cache-dir -r requirements.txt
+# Assume the Conda environment "keepaway" is already created
+RUN conda run -n keepaway /bin/bash -c "pip install --no-cache-dir -r requirements.txt"
+
+RUN conda run -n keepaway /bin/bash -c "pip install -e ."
+
+WORKDIR $TEAM_DIR/Pyrus2D/keepaway
+
+ENTRYPOINT ["/usr/local/bin/entrypoint.sh"]
+
+CMD ["conda", "run", "-n", "keepaway", "python3", "examples/training_agent.py --gc=3v2 --policy=handcoded --num_episodes=1000000"]
+

Makefile

@@ -0,0 +1,13 @@
+cleanall:
+	rm -fr keeper_Q*.gz taker_Q*.gz logs/* *.lock core core.* vgcore.* 
+	rm -fr *.lock console.log nohup.out *.dot *.xml
+	rm -fr keeper_Q*.log
+	rm -fr taker_Q*.log
+	rm -fr python_debug.log
+	rm -fr player*
+	rm -fr keepers-*.log
+	rm -fr takers-*.log
+	rm -fr NA-*.log
+	rm -fr *.rcg *.rcl
+	./kill.sh 
+	killall -q keepaway_player 1>/dev/null 2>&1

README.md

@@ -1,121 +1,122 @@
-# PYRUS2D
+# Keepaway-Python
 
-## Robocup Soccer Simmulation 2D Python Base Code
 
+This project offers a python implementation of the RoboCup keepaway environment, designed for deep reinforcement learning research. The environment is based on the [CYRUS](https://arxiv.org/abs/2211.08585) Python based RoboCup 2D soccer simulator, enabling researchers and developers to test and develop multi-agent reinforcement learning algorithms.
 
-PYRUS2D is the first Python base code (sample team) for RoboCup Soccer 2D Simulator.
-This project is implemented by members of CYRUS soccer simulation 2D team.
-By using this project, a team includes 11 players and one coach can connect to RoboCup Soccer Server and play a game.
-Also, researchers can use the trainer to control the server for training proposes.
+The published [paper](https://www.cs.utexas.edu/~pstone/Papers/bib2html/b2hd-AB05.html) and [code](https://github.com/tjpalmer/keepaway) outlines the motivation for keepaway for reinforcement learning and some initial research results using the environment.
+
+
+To get started with this project, follow the instructions in the [Quick Start Guide](##Quick-Start-Guide).
+
+### Using Docker
+
+This is the recommended way to avoid incompatible package versions. Instructions are available [here](docs/docker.md).
+
+
+
+<!-- * [RoboCup ](https://www.robocup.org/) [Soccer Simulation 2D League](https://rcsoccersim.github.io/) 2D simulation League
+* [Server documentation](https://rcsoccersim.readthedocs.io/) RCSSServer Documentation -->
 
 ---
-## Dependencies
+## Quick Start Guide
 
+### On your computer
 
+#### 1. Install required dependencies 
+#### Linux
 
-### RoboCup Soccer Simulation Server and Monitor
+This code has only been tested on Ubuntu 20.04, which rcssserver and rcssmonitor is supported.
 
 Install rcssserver and rcssmonitor (soccer window for debugging proposes)
 
 - rcssserver: [https://github.com/rcsoccersim/rcssserver](https://github.com/rcsoccersim/rcssserver)
 - rcssmonitor: [https://github.com/rcsoccersim/rcssmonitor](https://github.com/rcsoccersim/rcssmonitor)
 - soccer window: [https://github.com/helios-base/soccerwindow2](https://github.com/helios-base/soccerwindow2)
 
-### Python requirements
+#### 2. Install Python requirements
 
-- Python v3.9
-- coloredlogs==15.0.1
-- pyrusgeom==0.1.2
-- scipy==1.10.1
+- Python version 3.11
 
-```bash
+```
 pip install -r requirements.txt
 ```
 
----
-
-## Quick Start
+#### 3. Installing keepaway
 
-### Run team (11 players and 1 coach)
 
-To run the team there are some options,
+Create a Virtual Environment [virtual environment](https://docs.python.org/3/tutorial/venv.html):
 
-- running agents(players and coach execute separately)
-
-```bash
-cd Pyrus2D
-./start.sh
+```shell
+python3 -m venv keepaway-env
+source keepaway-env/bin/activate
 ```
 
-- running the agents by Python (or running them separately in PyCharm)
+Clone and checkout the release branch
 
-```bash
-cd Pyrus2D
-python team/main_player.py (11 times)
-python team/main_coach.py
+```shell
+git clone https://github.com/aabayomi/keepaway-python.git
+cd keepaway-python
+git checkout keepaway-release
 ```
 
-- running the agents by using one Python main process
+Install keepaway as local package
 
-```bash
-cd Pyrus2D
-python main.py
+```shell
+pip install -e .
+```
+#### 4. Run baseline policy 
+```shell
+python3 -m examples.test_random_agent
 ```
 
 
-## Start team by arguments
-
-
-To modify team configuration, you can pass the arguments or update ```team_config.py```.
-
-Configurations are listed bellow:
-
-```bash
-# To change the teamname (default is PYRUS):
--t|--teamname TeamName 
+## Training agents for keepaway
 
-# Determines if the connecting player is goalie or not. (take no parameters)
--g|--goalie 
+<!-- #### Baseline Policy
 
-# To change Output of loggers(Default is std):
-#   - std: loggers print data on the terminal -> std output
-#   - textfile: loggers write on the files based on players unum. (player-{unum}.txt, player-{unum}.err)
--o|--out [std|textfile] 
+To run and test one of the three baseline polices by [Gregory Kuhlmann and Peter Stone](https://www.cs.utexas.edu/~pstone/Papers/bib2html/b2hd-AB05.html). Follow the command below. 
 
-# change the host(serve) ip adderess. (defualt is localhost)
--H|--host new_ip_address
+All three polices can be found in the directory -->
 
-# change the player port connection. (default is 6000)
--p|--player-port new_port
+<!-- ```shell
+keepaway/
+    examples/
+``` -->
 
-# change the coach port connection. (default is 6002) 
--P|--coach-port new_port
+<!-- For example running the Handcoded policy  -->
 
-# change the trainer port connection. (default is 6001)
---trainer-port new_port
+<!-- ```
+cd keepaway
+python3 examples/test_handcoded_agent.py
+``` -->
 
-```
+<!-- #### Train Custom Policy -->
 
----
+<!-- To train a custom multi-agent policy you can edit the policies directory.
+```shell
+keepaway/
+    examples/
+``` -->
 
-## Useful links
+<!-- Policies
 
-- CYRUS team: [https://cyrus2d.com/](https://cyrus2d.com/)
-- RoboCup: [https://www.robocup.org/](https://www.robocup.org/)
-- Soccer Simulation 2D League: [https://rcsoccersim.github.io/](https://rcsoccersim.github.io/)
-- Server documentation: [https://rcsoccersim.readthedocs.io/](https://rcsoccersim.readthedocs.io/)
+```shell
+keepaway/
+    envs/
+        policies/
+``` -->
 
-## Related Papers
-
-- Zare N, Amini O, Sayareh A, Sarvmaili M, Firouzkouhi A, Rad SR, Matwin S, Soares A. Cyrus2D Base: Source Code Base for RoboCup 2D Soccer Simulation League. InRoboCup 2022: Robot World Cup XXV 2023 Mar 24 (pp. 140-151). Cham: Springer International Publishing. [link](https://arxiv.org/abs/2211.08585)
-- Zare N, Sarvmaili M, Sayareh A, Amini O, Matwin S, Soares A. Engineering Features to Improve Pass Prediction in Soccer Simulation 2D Games. InRobot World Cup 2022 (pp. 140-152). Springer, Cham. [link](https://www.researchgate.net/profile/Nader-Zare/publication/352414392_Engineering_Features_to_Improve_Pass_Prediction_in_Soccer_Simulation_2D_Games/links/60c9207fa6fdcc0c5c866520/Engineering-Features-to-Improve-Pass-Prediction-in-Soccer-Simulation-2D-Games.pdf)
-- Zare N, Amini O, Sayareh A, Sarvmaili M, Firouzkouhi A, Matwin S, Soares A. Improving Dribbling, Passing, and Marking Actions in Soccer Simulation 2D Games Using Machine Learning. InRobot World Cup 2021 Jun 22 (pp. 340-351). Springer, Cham. [link](https://www.researchgate.net/profile/Nader-Zare/publication/355680673_Improving_Dribbling_Passing_and_Marking_Actions_in_Soccer_Simulation_2D_Games_Using_Machine_Learning/links/617971b0a767a03c14be3e42/Improving-Dribbling-Passing-and-Marking-Actions-in-Soccer-Simulation-2D-Games-Using-Machine-Learning.pdf)
-- Akiyama, H., Nakashima, T.: Helios base: An open source package for the robocup soccer 2d simulation. In Robot Soccer World Cup 2013 Jun 24 (pp. 528-535). Springer, Berlin, Heidelberg.
+<!-- #### To view the running player in the soccerwindow/rcssmonitor
 
+``` 
+ctrl + C
+```
 
-## Cite and Support
+#### Viewing Logs
 
-Please don't forget to cite our papers and star our GitHub repo if you haven't already!
-## Contributing
+The gameserver logs, error and keepaway logs are in the this directory
 
-Pull requests are welcome. For major changes, please open an issue first to discuss what you would like to change.
+```shell
+keepaway/
+    logs/
+``` -->