Autonomous Car Navigation Using PPO

This work was inspired by this project: "DRL-Nav: Autonomous UAV Navigation via Deep Reinforcement Learning Using PPO". Here the steering action is continuous and the car is trained to navigate through an AirSim environment: AirSimNH, which can be downloaded here

This repository contains an implementation of Proximal Policy Optimization (PPO) for autonomous navigation in a neighborhood environment with a car. The goal is to make the car learn to drive in its environment while avoiding collisions.

Contents

• Overview
  • Inputs
  • Actions
  • Model structure
• Environment setup to run the codes
• How to run the training?
• How to run the pretrained model?
• Inference on the final trained model

Overview:

In this project are present all the files necessary to make the PPO algorithm work in three main mode: sampling the action from a Normal distribution, a Beta distribution and a Flexible Beta distribution. Explainability is possible through GradCAM method.

Inputs:

images from the front camera of the car

Actions:

continuous domain, steering action between [-1,1]

Model structure:

Environment setup to run the codes

1. Clone the repository

git clone https://github.com/Pedrazzini/AutonomousDriveEP.git

2. From Anaconda command prompt, create a new conda environment

I recommend you to use Anaconda or Miniconda to create a virtual environment.

conda create -n drl_nav python==3.8

3. Install required libraries

Inside the main directory of the repo

conda activate drl_nav pip install -r requirements.txt

4. (Optional) Install Pytorch for GPU

You must have a CUDA supported NVIDIA GPU.

4. Edit settings.json

Content of the settings.json should be as below:

The setting.json file is located at Documents\AirSim folder.

json { "SeeDocsAt": "https://github.com/Microsoft/AirSim/blob/main/docs/settings.md", "SettingsVersion": 1.2, "LocalHostIp": "127.0.0.1", "SimMode": "Car", "ClockSpeed": 1, "ViewMode": "Fpv", "Vehicles": { "car0": { "VehicleType": "PhysXCar", "X": 0.0, "Y": 0.0, "Z": 0.0, "Yaw": 0.0 } }, "CameraDefaults": { "CaptureSettings": [ { "ImageType": 0, "Width": 50, "Height": 50, "FOV_Degrees": 120 }, { "ImageType": 2, "Width": 50, "Height": 50, "FOV_Degrees": 120 } ] } }

How to run the training?

Make sure you followed the instructions above to setup the environment.

1. Download the training environment

Go to the releases and download AirSimNH.zip. After downloading completed, extract it and follow the instruction on this page.

3. Now, you can open up environment's executable file and start the training

So, inside the repository python train.py

Inference on the final trained model:

Citation

Proximal Policy Optimization Algorithms by John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov.

Author

• Ernesto Pedrazzini

License

This project is licensed under the GNU Affero General Public License.

A u t o n o m o u s D r i v e E P