Multi Agent Deep Deterministic Policy Gradients MADDPG algorithm based agent for solving the multi agent Tennis Environment (Unity) as part of Udacity's Deep Reinforcement Nanodegree by Sayon Palit
- For this project we work with Unity's Tennis environment
-
In this environment, two agents control rackets to bounce a ball over a net. If an agent hits the ball over the net, it receives a reward of +0.1. If an agent lets a ball hit the ground or hits the ball out of bounds, it receives a reward of -0.01. Thus, the goal of each agent is to keep the ball in play.
-
The observation space consists of 8 variables corresponding to the position and velocity of the ball and racket. Each agent receives its own, local observation. Two continuous actions are available, corresponding to movement toward (or away from) the net, and jumping.
-
The task is episodic, and in order to solve the environment, your agents must get an average score of +0.5 (over 100 consecutive episodes, after taking the maximum over both agents). Specifically,
- After each episode, we add up the rewards that each agent received (without discounting), to get a score for each agent. This yields 2 (potentially
different) scores. We then take the maximum of these 2 scores. - This yields a single score for each episode.
- After each episode, we add up the rewards that each agent received (without discounting), to get a score for each agent. This yields 2 (potentially
The environment is considered solved, when the average (over 100 episodes) of those scores is at least +0.5.
To run the codes, follow the next steps:
- Create a new environment:
- Linux or Mac:
conda create --name drlnd python=3.6 source activate drlnd
- Windows:
conda create --name drlnd python=3.6 activate drlnd
- Perform a minimal install of OpenAI gym
- If using Windows,
- download swig for windows and add it the PATH of windows
- install Microsoft Visual C++ Build Tools
- then run these commands
pip install gym
- If using Windows,
- Download this repository or clone it
git clone https://github.com/sayonpalit2599/p3-collaboration-competition/
- Install the dependencies under the folder python/
cd python
pip install .
- Create an IPython kernel for the
drlnd
environment
python -m ipykernel install --user --name drlnd --display-name "drlnd"
-
Download the Unity Environment specific to your operating system
-
Start jupyter notebook from the root of this python codes
jupyter notebook
-
Once started, change the kernel through the menu
Kernel
>Change kernel
>drlnd
-
If necessary, inside the ipynb files, change the path to the unity environment appropriately
-
Follow the instructions in
Tennis-v2.0.ipynb
to train the agent and see the performance of the agent. -
It is recommended to use a GPU for running this code
- The current model solves the environment in approx ~1906 epsiodes on average with an
NVIDIA GTX1050TI
andIntel i8750H
.