Reinforcement learning approach of training physics based car which learns to avoid obstacles. Project was made in Unity, using MLAgents, reinfercement learning and C# scripts. The car recieves positive reward for driving into the goal and negative reward for driving into the walls, obstacles or for circling around.
#Overview
This is first project I've seen that takes on using Reinforcement learning for physics based car.
Car controller is based on wheel collider physics therefore it was tricky to train such car.
The main goal is for the car to reach certain point on the map while avoiding objects, obstacles and other cars.
Model was set to train on static obstacles for 5 millions steps. After succesfully training, it was time to move on to the randomly generated positions of the obstacles which took over 60 million steps to train perfectly
[INFO] MoveToGoal. Step: 4800000. Time Elapsed: 5777.964 s. Mean Reward: 97.196. Std of Reward: 20.315. Training.
[INFO] MoveToGoal. Step: 5050000. Time Elapsed: 5837.711 s. Mean Reward: 100.000. Std of Reward: 0.000. Training.
Final goal was to train model with dynamic obstacles (red cars). This process took over 200 million steps.
At each frame Neural Network was getting information about Vector3 coordinates of the car itself and the posotion of the goal area.
Also lidar was used in form of collision rays (Ray Perception sensor 3D) with such parametres:
Stacked vectors should be set to 1 in case of static obstacles and 4-6 in case of dynamic obstacles. The latter amount allows for NN to look at multiple states at the same time therefore seeing and predicting moves along with training progress.
Here's a video of model running on inference mode with fully trained model!