safe-control-gym

PyBullet CartPole and Quadrotor environments—with CasADi symbolic a priori dynamics—for learning-based control and RL

https://github.com/utiasdsl/safe-control-gym

Science Score: 59.0%

This score indicates how likely this project is to be science-related based on various indicators:

○
CITATION.cff file
✓
codemeta.json file
Found codemeta.json file
✓
.zenodo.json file
Found .zenodo.json file
✓
DOI references
Found 2 DOI reference(s) in README
✓
Academic publication links
Links to: arxiv.org, ieee.org
✓
Committers with academic emails
10 of 20 committers (50.0%) from academic institutions
○
Institutional organization owner
○
JOSS paper metadata
○
Scientific vocabulary similarity
Low similarity (14.9%) to scientific vocabulary

Keywords

cartpole casadi control gym pybullet quadcopter quadrotor reinforcement-learning robotics robustness safety symbolic

Last synced: 6 months ago · JSON representation

Repository

PyBullet CartPole and Quadrotor environments—with CasADi symbolic a priori dynamics—for learning-based control and RL

Basic Info

Host: GitHub
Owner: utiasDSL
License: mit
Language: Python
Default Branch: main
Homepage: https://www.dynsyslab.org/safe-robot-learning/
Size: 208 MB

Statistics

Stars: 755
Watchers: 12
Forks: 144
Open Issues: 7
Releases: 5

Topics

cartpole casadi control gym pybullet quadcopter quadrotor reinforcement-learning robotics robustness safety symbolic

Created over 4 years ago · Last pushed 6 months ago

Metadata Files

Readme License Citation

safe-control-gym

Physics-based CartPole and Quadrotor Gym environments (using PyBullet) with symbolic a priori dynamics (using CasADi) for learning-based control, and model-free and model-based reinforcement learning (RL).

These environments include (and evaluate) symbolic safety constraints and implement input, parameter, and dynamics disturbances to test the robustness and generalizability of control approaches. [PDF]

problem illustration

bibtex @article{brunke2021safe, title={Safe Learning in Robotics: From Learning-Based Control to Safe Reinforcement Learning}, author={Lukas Brunke and Melissa Greeff and Adam W. Hall and Zhaocong Yuan and Siqi Zhou and Jacopo Panerati and Angela P. Schoellig}, journal = {Annual Review of Control, Robotics, and Autonomous Systems}, year={2021}, url = {https://arxiv.org/abs/2108.06266}}

To reproduce the results in the article, see branch ar.

bibtex @article{yuan2021safecontrolgym, author={Yuan, Zhaocong and Hall, Adam W. and Zhou, Siqi and Brunke, Lukas and Greeff, Melissa and Panerati, Jacopo and Schoellig, Angela P.}, journal={IEEE Robotics and Automation Letters}, title={Safe-Control-Gym: A Unified Benchmark Suite for Safe Learning-Based Control and Reinforcement Learning in Robotics}, year={2022}, volume={7}, number={4}, pages={11142-11149}, doi={10.1109/LRA.2022.3196132}}

To reproduce the results in the article, see branch submission.

Install on Ubuntu/macOS

Clone repo

bash git clone https://github.com/utiasDSL/safe-control-gym.git cd safe-control-gym

(optional) Create a `conda` environment

Create and access a Python 3.10 environment using conda

bash conda create -n safe python=3.10 conda activate safe

Install

Install the safe-control-gym repository

bash python -m pip install --upgrade pip python -m pip install -e .

Note

You may need to separately install gmp, a dependency of pycddlib:

bash conda install -c anaconda gmp

bash sudo apt-get install libgmp-dev

#### (optional) Additional requirements for MPC

You may need to separately install acados for fast MPC implementations.

To build and install acados, see their installation guide.
To set up the acados python interface, check out these installtion steps.

Architecture

Overview of safe-control-gym's API:

block diagram

Configuration

config

Getting Started

Familiarize with APIs and environments with the scripts in examples/

3D Quadrotor Lemniscate Trajectory Tracking with PID

bash cd ./examples/ # Navigate to the examples folder python3 pid/pid_experiment.py \ --algo pid \ --task quadrotor \ --overrides \ ./pid/config_overrides/quadrotor_3D/quadrotor_3D_tracking.yaml

systems trajectory

Cartpole Stabilization with LQR

bash cd ./examples/ # Navigate to the examples folder python3 lqr/lqr_experiment.py \ --algo lqr \ --task cartpole \ --overrides \ ./lqr/config_overrides/cartpole/cartpole_stabilization.yaml \ ./lqr/config_overrides/cartpole/lqr_cartpole_stabilization.yaml

2D Quadrotor Trajectory Tracking with PPO

bash cd ./examples/rl/ # Navigate to the RL examples folder python3 rl_experiment.py \ --algo ppo \ --task quadrotor \ --overrides \ ./config_overrides/quadrotor_2D/quadrotor_2D_track.yaml \ ./config_overrides/quadrotor_2D/ppo_quadrotor_2D.yaml \ --kv_overrides \ algo_config.training=False

Verbose API Example

bash cd ./examples/ # Navigate to the examples folder python3 no_controller/verbose_api.py \ --task cartpole \ --overrides no_controller/verbose_api.yaml

List of Implemented Controllers

List of Implemented Safety Filters

Performance

We compare the sample efficiency of safe-control-gym with the original OpenAI Cartpole and PyBullet Gym's Inverted Pendulum, as well as gym-pybullet-drones. We choose the default physic simulation integration step of each project. We report performance results for open-loop, random action inputs. Note that the Bullet engine frequency reported for safe-control-gym is typically much finer grained for improved fidelity. safe-control-gym quadrotor environment is not as light-weight as gym-pybullet-drones but provides the same order of magnitude speed-up and several more safety features/symbolic models.

| Environment | GUI | Control Freq. | PyBullet Freq. | Constraints & Disturbances^ | Speed-Up^^ | | :------------------------: | :----: | :------------: | :-------------: | :-------------------------------: | :-------------: | | Gym cartpole | True | 50Hz | N/A | No | 1.16x | | InvPenPyBulletEnv | False | 60Hz | 60Hz | No | 158.29x | | cartpole | True | 50Hz | 50Hz | No | 0.85x | | cartpole | False | 50Hz | 1000Hz | No | 24.73x | | cartpole | False | 50Hz | 1000Hz | Yes | 22.39x | | | | | | | | | gym-pyb-drones | True | 48Hz | 240Hz | No | 2.43x | | gym-pyb-drones | False | 50Hz | 1000Hz | No | 21.50x | | quadrotor | True | 60Hz | 240Hz | No | 0.74x | | quadrotor | False | 50Hz | 1000Hz | No | 9.28x | | quadrotor | False | 50Hz | 1000Hz | Yes | 7.62x |

^ Whether the environment includes a default set of constraints and disturbances

^^ Speed-up = Elapsed Simulation Time / Elapsed Wall Clock Time; on a 2.30GHz Quad-Core i7-1068NG7 with 32GB 3733MHz LPDDR4X; no GPU

Run Tests and Linting

Tests can be run locally by executing: bash python3 -m pytest ./tests/ # Run all tests

Linting can be run locally with: bash pre-commit install # Install the pre-commit hooks pre-commit autoupdate # Auto-update the version of the hooks pre-commit run --all # Run the hooks on all files

References

Brunke, L., Greeff, M., Hall, A. W., Yuan, Z., Zhou, S., Panerati, J., & Schoellig, A. P. (2022). Safe learning in robotics: From learning-based control to safe reinforcement learning. Annual Review of Control, Robotics, and Autonomous Systems, 5, 411-444.
Yuan, Z., Hall, A. W., Zhou, S., Brunke, L., Greeff, M., Panerati, J., & Schoellig, A. P. (2022). safe-control-gym: A unified benchmark suite for safe learning-based control and reinforcement learning in robotics. IEEE Robotics and Automation Letters, 7(4), 11142-11149.

Related Open-source Projects

gym-pybullet-drones: single and multi-quadrotor environments
stable-baselines3: PyTorch reinforcement learning algorithms
bullet3: multi-physics simulation engine
gym: OpenAI reinforcement learning toolkit
casadi: symbolic framework for numeric optimization
safety-gym: environments for safe exploration in RL
realworldrl_suite: real-world RL challenge framework
gym-marl-reconnaissance: multi-agent heterogeneous (UAV/UGV) environments

University of Toronto's Dynamic Systems Lab / Vector Institute for Artificial Intelligence

Owner

Name: Dynamic Systems Lab
Login: utiasDSL
Kind: organization

Website: http://www.dynsyslab.org/
Repositories: 4
Profile: https://github.com/utiasDSL

GitHub Events

Total

Create event: 3
Issues event: 20
Release event: 1
Watch event: 142
Delete event: 8
Issue comment event: 53
Push event: 22
Pull request review event: 101
Pull request review comment event: 87
Pull request event: 41
Fork event: 18

Last Year

Create event: 3
Issues event: 20
Release event: 1
Watch event: 142
Delete event: 8
Issue comment event: 53
Push event: 22
Pull request review event: 101
Pull request review comment event: 87
Pull request event: 41
Fork event: 18

Committers

Last synced: 9 months ago

All Time

Total Commits: 448
Total Committers: 20
Avg Commits per committer: 22.4
Development Distribution Score (DDS): 0.663

Past Year

Commits: 21
Committers: 6
Avg Commits per committer: 3.5
Development Distribution Score (DDS): 0.286

Top Committers

Name	Email	Commits
Federico-PizarroBejarano	f**o@m**a	151
Jacopo Panerati	j**i@u**a	146
Catherine Glossop	c**p@v**i	54
Adam Hall	a**l@g**m	45
Justin-Yuan	j**n@g**m	28
catglossop	c**p@m**a	4
Amrit	a**k@v**i	3
pizarrob	p**b@q**i	2
Jakob Thumm	j**m@i**e	2
Mingxuan Che	5****e	2
SiQi Zhou	s**u@r**a	2
Andrea Ghezzi	9****1	1
Emmanuel Ferdman	e**n@g**m	1
JakobThumm	j**m@w**e	1
Jon Goikoetxea	1****o	1
Lukas Brunke	l**e@t**e	1
pizarrob	p**b@v**l	1
pizarrob	p**b@v**l	1
Tsung Yuan Tseng	3****n	1
melissagreeff204	m**f@m**a	1

Committer Domains (Top 20 + Academic)

mail.utoronto.ca: 3 vectorinstitute.ai: 2 tu-harburg.de: 1 robotics.utias.utoronto.ca: 1 in.tum.de: 1 q.vectorinstitute.ai: 1 utoronto.ca: 1

Issues and Pull Requests

Last synced: 6 months ago

All Time

Total issues: 30
Total pull requests: 111
Average time to close issues: 3 months
Average time to close pull requests: about 1 month
Total issue authors: 24
Total pull request authors: 21
Average comments per issue: 3.7
Average comments per pull request: 0.96
Merged pull requests: 71
Bot issues: 0
Bot pull requests: 0

Past Year

Issues: 11
Pull requests: 30
Average time to close issues: 2 months
Average time to close pull requests: about 1 month
Issue authors: 8
Pull request authors: 7
Average comments per issue: 3.36
Average comments per pull request: 1.2
Merged pull requests: 14
Bot issues: 0
Bot pull requests: 0

View more stats

Top Authors

Issue Authors

Federico-PizarroBejarano (5)
zcase (4)
JakobThumm (2)
adamhall (2)
SapanaChaudhary (2)
MingxuanChe (2)
aghezz1 (1)
Hongyu-ZHOU (1)
hedaniel7 (1)
XilunZhangRobo (1)
Tianle-hub (1)
Jinyi6 (1)
HP-CAO (1)
swuuu (1)
tfarger (1)

Pull Request Authors

Federico-PizarroBejarano (45)
JacopoPan (20)
catgloss (11)
adamhall (10)
aghezz1 (8)
Justin-Yuan (7)
MingxuanChe (5)
amrit110 (2)
middleyuan (2)
emmanuel-ferdman (2)
allenapplehead (2)
migooll (2)
spencerteetaert (2)
catglossop (2)
hannakrasowski (1)

Top Labels

Issue Labels

question (14) bug (9) enhancement (4) competition (2) help wanted (2) documentation (1) invalid (1)

Pull Request Labels

enhancement (62) bug (26) documentation (9) linting (8) new controller (6) competition (6) merge when reviewed (2) help wanted (1)

Packages

Total packages: 2
Total downloads: unknown

Total dependent packages: 0
(may contain duplicates)
Total dependent repositories: 0
(may contain duplicates)
Total versions: 8

proxy.golang.org: github.com/utiasdsl/safe-control-gym

Documentation: https://pkg.go.dev/github.com/utiasdsl/safe-control-gym#section-documentation
License: mit
Latest release: v1.0.0
published almost 2 years ago

Versions: 4
Dependent Packages: 0
Dependent Repositories: 0

Rankings

Dependent packages count: 6.5%

Average: 6.7%

Dependent repos count: 6.9%

Last synced: 6 months ago

proxy.golang.org: github.com/utiasDSL/safe-control-gym

Documentation: https://pkg.go.dev/github.com/utiasDSL/safe-control-gym#section-documentation
License: mit
Latest release: v1.0.0
published almost 2 years ago

Versions: 4
Dependent Packages: 0
Dependent Repositories: 0

Rankings

Dependent packages count: 6.5%

Average: 6.7%

Dependent repos count: 6.9%

Last synced: 6 months ago

Dependencies

.github/workflows/push.yml actions

actions/checkout v3 composite
actions/setup-python v3 composite

pyproject.toml pypi

Mosek ^10.0.18
PyYAML ^6.0
casadi ^3.6.0
cvxpy ^1.1.18
dict-deep ^4.1.2
gpytorch ^1.6.0
gymnasium ^0.28
imageio ^2.14.1
matplotlib ^3.5.1
munch ^2.5.0
numpy ^1.22.1
pre-commit ^3.3.2
pybullet ^3.2.0
pytest ^7.2.2
python ^3.10
pytope ^0.0.4
scikit-learn ^1.3.0
scikit-optimize ^0.9.0
tensorboard ^2.12.0
termcolor ^1.1.0
torch ^1.10.2

setup.py pypi

Mosek *
casadi *
cvxpy *
dict-deep *
gpytorch *
gymnasium *
imageio *
matplotlib *
munch *
numpy *
pre-commit *
pybullet *
pytest *
pytope *
pyyaml *
scikit-learn *
scikit-optimize *
tensorboard *
termcolor *
torch *

safe-control-gym

Science Score: 59.0%

Keywords

Repository

Basic Info

Statistics

Topics

Metadata Files

README.md

safe-control-gym

Install on Ubuntu/macOS

Clone repo

(optional) Create a conda environment

Install

Note

Architecture

Configuration

Getting Started

3D Quadrotor Lemniscate Trajectory Tracking with PID

Cartpole Stabilization with LQR

2D Quadrotor Trajectory Tracking with PPO

Verbose API Example

List of Implemented Controllers

List of Implemented Safety Filters

Performance

Run Tests and Linting

References

Related Open-source Projects

Owner

GitHub Events

Total

Last Year

Committers

All Time

Past Year

Top Committers

Committer Domains (Top 20 + Academic)

Issues and Pull Requests

All Time

Past Year

Top Authors

Issue Authors

Pull Request Authors

Top Labels

Issue Labels

Pull Request Labels

Packages

proxy.golang.org: github.com/utiasdsl/safe-control-gym

Rankings

proxy.golang.org: github.com/utiasDSL/safe-control-gym

Rankings

Dependencies

(optional) Create a `conda` environment