This repository contains material from the second Udacity DRL procjet and the coding exercice DDPG-pendulum.
In this project, I trained a DDPG agent to solve two types of environment.
First the Reacher environment, a double-jointed arm can move to target locations. A reward of +0.1 is provided for each step that the agent's hand is in the goal location. Thus, the goal of your agent is to maintain its position at the target location for as many time steps as possible.
The observation space consists of 33 variables corresponding to position, rotation, velocity, and angular velocities of the arm. Each action is a vector with four numbers, corresponding to torque applicable to two joints. Every entry in the action vector should be a number between -1 and 1.
Second, the Crawler environment.
In this continuous control environment, the goal is to teach a creature with four legs to walk forward without falling.
An environment is considered solved, when an average score of +30 over 100 consecutive episodes, and over all agents is obtained.
To set up your python environment to run the code in this repository, follow the instructions below.
-
Create (and activate) a new environment with Python 3.9.
- Linux or Mac:
conda create --name drlnd source activate drlnd
- Windows:
conda create --name drlnd activate drlnd
-
Follow the instructions in Pytorch web page to install pytorch and its dependencies (PIL, numpy,...). For Windows and cuda 11.6
conda install pytorch torchvision torchaudio cudatoolkit=11.6 -c pytorch -c conda-forge
-
Follow the instructions in this repository to perform a minimal install of OpenAI gym.
- Install the box2d environment group by following the instructions here.
pip install gym[box2d]
-
Follow the instructions in second Udacity DRL procjet to get the environment.
-
Clone the repository, and navigate to the
python/
folder. Then, install several dependencies.
git clone https://github.com/eljandoubi/DDPG-for-continuous-control.git
cd DDPG-for-continuous-control/python
pip install .
- Create an IPython kernel for the
drlnd
environment.
python -m ipykernel install --user --name drlnd --display-name "drlnd"
- Before running code in a notebook, change the kernel to match the
drlnd
environment by using the drop-downKernel
menu.
You can train and/or inference an environment by following instructions in its notebook.
The implementation and resultats are discussed in the report.