Lero Learning Control

The "Lero Learning Control" repository is a dedicated hub for sharing experimental source code and results related to learning control. This repository is part of a larger effort funded by Science Foundation Ireland, through "Grant No," to support software research and development at Lero - University College Cork, Ireland.

The repository provides a centralized and replicable location for researchers, practitioners, and other interested parties to access and utilize the experimental source code and results contained within. This allows for improved collaboration and greater transparency in the field of learning control, facilitating further research and experimentation.

If you find the "Lero Learning Control" repository helpful in your own publications, experiments, or other research, the creators kindly request that you consider citing their work in your own publications. This citation will help to promote and support the continued development of this valuable resource for the learning control community: @misc{leroLearningControl2021, title={{Lero Learning Control Repository}}, author={Yves Sohege, Gregory Provan}, howpublished = {\url{https://github.com/YSohege/lero-learning-control}}, year={2021} }

List of Implemented Environments and Controllers

2 Dimensional Quadcopter Environment - based on Safe Control Gym

• Constraint Barrier Functions - CBF
• Linear Quadratic Regulator - LQR
• Model Predictive Controller - MPC
• Proportional Integral Derivative - PID
• Proximal Policy Optimization - PPO
• Robust Adversarial Reinforcement Learning - RARL
• Soft Actor Critic - SAC
• SafeExplorer
• Randomized Blended MPC Control - RBCMPC
• Randomized Blended PID COntrol - RBCPID

3 Dimensional Quadcopter Environment

• Switched PID Control
• Multiple Model Adaptive PID Control - MMACPID
• Randomized Blended PID Control
• Fixed Distribution Randomized Blended PID Control
• Uniform Randomized Blended Control
• Reinforcement Learning Randomized Blended PID Control - RLRBCPID

List of Tasks

Tasks require a Controller / Environment pair and return an performance metric after completion. 1) Blended Control Task (3D Quadcopter / RBCPID Controller)- Set blending weigth at each step - used for reinforcement learning 2) Fixed Distribution Blended Control Task (3D Quadcopter / Fixed Distribution RBCPID Controller) - Evaluate the tracking performance of RBCPID using a fixed distribution 3) Optimal Switching Control Task (3D Quadcopter / Switched PID Controller) - Evaluate the tracking performance of Switching Control with no fault identification delay (optimal baseline) 4) RL Distribuition for Blended Control (3D Quadcopter / RLRBCPID Controller) - Learn optimal randomized blended control distribution 5) Simple MMAC Quadcopter tracking (3D Quadcopter / MMACPID Controller) - Simple tractory tracking task 6) Uniform RBCPID Quadcopter tracking (3D Quadcopter / Uni RBCPID Controller) - Evaluate the tracking performance of uniformly sampled blending weights.

List of Tuning Algorithms

Tuning algorithms execute Tasks repeadedly to maximize the achieved performance through controller parameter optimization.

1) Clustered Particle Filtering (3D Quadcopter / PID Controller) - CPF 2) Reinforcement Learning Blended Control (3D Quadcopter / RLRBCPID Controller) - RL Blending 3) Particle Swarm Optimization(3D Quadcopter / PID Controller) - PSO

List of Example Experiments

Experiments run the Tuning algorithms or Tasks in a systematic way based on their configurations. The experiments are repeatable and all random seeds are fixed in the config files. 1) 2D Quadcopter - RBCMPC versus RBCPID trajectory tracking performance 2) 3D Quadcopter - Switched Control 3) 3D Quadcopter - Data Generation under various fault and disturbances 4) 3D Quadcopter - MPC Test 5) 3D Quadcopter - Controller Tuning using Clustered Particle Filtering (CPF) 6) Execution speed test - numpy versus torch / CPU versus GPU 7) 3D Quadcopter - Simple MPC control for trajectory tracking

Setup and Dependencies

Windows

1) Install Python 3.x from the official Python website: https://www.python.org/downloads/windows/ 2) Open Command Prompt or PowerShell as an administrator. 3) Navigate to the directory where you want to install Lero Learning Control. 4) Clone the repository: git clone https://github.com/YSohege/lero-learning-control.git 5) Navigate to the repository: cd lero_learning_control 6) Install the required libraries: python setup.py install

Linux

1) Install Python 3.x using your system's package manager. For example, on Ubuntu, you can use the following command: sudo apt-get install python3 2) Open a terminal window. 3) Navigate to the directory where you want to install Lero Learning Control. 4) Clone the repository: git clone https://github.com/YSohege/lero-learning-control.git 5) Navigate to the repository: cd lero_learning_control 6) Install the required libraries: python setup.py install Note that you may need to use python3 / pip3 instead of python / pip on some Linux distributions.