minimal-gym-rotor

Minimal OpenAI Gym-based environments for a quadrotor UAV

Learn by Doing

This repository contains OpenAI Gym-based environments for low-level control of quadrotor unmanned aerial vehicles (UAVs). This repo is designed to serve as an educational platform for those interested in building Gym-based environments. To better understand What Deep RL Do, see OpenAI Spinning UP. Please feel free to create new issues or pull requests for any suggestions and corrections.

Installation

It is recommended to create Anaconda environment with Python 3. The official installation guide is available here. Also, Visual Studio Code in Anaconda Navigator is highly recommended.

1. Open your Anaconda Prompt and install major packages.

conda install -c conda-forge gymnasium
conda install pytorch torchvision torchaudio pytorch-cuda=11.8 -c pytorch -c nvidia
conda install -c conda-forge vpython

Check out Gym, Pytorch, and Vpython.

2. Clone the repositroy.

git clone https://github.com/BeomyeolYu/minimal-gym-rotor.git

Environments

Consider a quadrotor UAV below:

The position and the velocity of the quadrotor are represented by $x \in \mathbb{R}^3$ and $v \in \mathbb{R}^3$, respectively. The attitude is defined by the rotation matrix $R \in SO(3) = \lbrace R \in \mathbb{R}^{3\times 3} | R^T R=I_{3\times 3}, \mathrm{det}[R]=1 \rbrace$, that is the linear transformation of the representation of a vector from the body-fixed frame $\lbrace \vec b_{1},\vec b_{2},\vec b_{3} \rbrace$ to the inertial frame $\lbrace \vec e_1,\vec e_2,\vec e_3 \rbrace$. The angular velocity vector is denoted by $\Omega \in \mathbb{R}^3$. From the thrust of each motor $(T_1,T_2,T_3,T_4)$, the total thrust $f = \sum{}_{i=1}^{4} T_i \in \mathbb{R}$ and the total moment $M \in \mathbb{R}^3$ resolved in the body-fixed frame can be represented.

Env IDs	Description
Quad-v0	The state and the action are given by $s = (e_x, e_v, R, \Omega)$ and $a = (T_1, T_2, T_3, T_4)$.

where the error terms $e_x, e_v$, and $e_\Omega$ represent the errors in position, velocity, and angular velocity, respectively.

Training RL Agents

If you're interested in training RL agents in the quadrotor environments provided here, we recommend visiting github.com/fdcl-gwu/gym-rotor. This companion repository focuses on training RL agents using PyTorch implementations of DDPG and TD3 and offers additional resources for in-depth experimentation.

Reference:

• https://github.com/openai/gym
• https://github.com/ethz-asl/reinmav-gym