Data

Tools

Indexes

Applications

Experiments

Open Source Science

drone-control-using-reinforcement-learning

Control drone with ppo in gym-pybullet-drones

https://github.com/phuongboi/drone-control-using-reinforcement-learning

Science Score: 44.0%

This score indicates how likely this project is to be science-related based on various indicators:

  • ✓
    CITATION.cff file
    Found CITATION.cff file
  • ✓
    codemeta.json file
    Found codemeta.json file
  • ✓
    .zenodo.json file
    Found .zenodo.json file
  • ○
    DOI references
  • ○
    Academic publication links
  • ○
    Committers with academic emails
  • ○
    Institutional organization owner
  • ○
    JOSS paper metadata
  • ○
    Scientific vocabulary similarity
    Low similarity (8.2%) to scientific vocabulary

Keywords

gym ppo quacopter reinforcement-learning
Last synced: 6 months ago · JSON representation ·

Repository

Control drone with ppo in gym-pybullet-drones

Basic Info
  • Host: GitHub
  • Owner: phuongboi
  • License: mit
  • Language: Python
  • Default Branch: main
  • Homepage:
  • Size: 25.1 MB
Statistics
  • Stars: 18
  • Watchers: 1
  • Forks: 2
  • Open Issues: 0
  • Releases: 0
Topics
gym ppo quacopter reinforcement-learning
Created about 2 years ago · Last pushed over 1 year ago
Metadata Files
Readme License Citation

README.md

Control drone in gym-pybullet-drones using ppo

Hovering a quacopter with some predefined position using gym-pybullet-drones env with PPO algorithm from PPO-PyTorch

25/09/2024 Update drone racing

  • Refer to my recent project on drone racing in gym pybulet #### 06/09/2024 Update fly through the gate
  • I test FlyThruGateAvitary environment with PPO with some modify in reward function. I created a gate model with Tinkercad to and add pybullet.
  • To train: python train_thrugate.py, to test: python test_thrugate.py #### 20/02/2024 Note about ppo implementation
  • Recently, I figure out the frustration of drone at hover position may come from fixed action_std of this PPO implementation, they setting action_std_init = 0.6 and decay this value during training time. ~~In inference mode, there is no mechanism to reduce or remove this variance, so control output this vary all the time.~~ I look at some other implementation of Soft Actor Critic, they use one more layer to learn action std beside action mean. #### 13/01/2024 Update hovering with some constrains
  • Add some contrains to naive reward, drone look more stable at hover position, reference from paper #### 30/12/2023 Update training result #### 28/12/2023 Init commit
  • Change reward function, compute terminate ##### Fly through the gate

alt text

Hover at (0, 0, 1) position

alt text

Hover at (0, 1, 1) position

alt text

How to use

  • Follow author's guide to install gym-pybullet-drones environment
  • Training python train_hover.py
  • Test pretrained model python test_hover.py

References

  • https://github.com/utiasDSL/gym-pybullet-drones/
  • https://github.com/nikhilbarhate99/PPO-PyTorch
  • Schulman, John, et al. "Proximal policy optimization algorithms." arXiv preprint arXiv:1707.06347 (2017).
  • https://web.stanford.edu/class/aa228/reports/2019/final62.pdf

Owner

  • Name: phuongboi
  • Login: phuongboi
  • Kind: user
  • Location: Vietnam

software engineer

Citation (CITATION.cff) 

cff-version: 1.2.0
preferred-citation:
  type: article
  authors:
  - family-names: "Panerati"
    given-names: "Jacopo"
    orcid: "https://orcid.org/0000-0003-2994-5422"
  - family-names: "Zheng"
    given-names: "Hehui"
    orcid: "https://orcid.org/0000-0002-4977-0220"
  - family-names: "Zhou"
    given-names: "SiQi"
  - family-names: "Xu"
    given-names: "James"
  - family-names: "Prorok"
    given-names: "Amanda"
    orcid: "https://orcid.org/0000-0001-7313-5983"
  - family-names: "Schoellig"
    given-names: "Angela P."
    orcid: "https://orcid.org/0000-0003-4012-4668"
  doi: "10.0000/00000"
  journal: "2021 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)"
  month: 1
  start: 1 # First page number
  end: 8 # Last page number
  title: "Learning to Fly---a Gym Environment with PyBullet Physics for Reinforcement Learning of Multi-agent Quadcopter Control"
  issue: 1
  volume: 1
  year: 2021

GitHub Events

Total
  • Watch event: 11
Last Year
  • Watch event: 11

Committers

Last synced: about 2 years ago

All Time
  • Total Commits: 17
  • Total Committers: 1
  • Avg Commits per committer: 17.0
  • Development Distribution Score (DDS): 0.0
Past Year
  • Commits: 17
  • Committers: 1
  • Avg Commits per committer: 17.0
  • Development Distribution Score (DDS): 0.0
Top Committers
Name Email Commits
phuongboi k****4@g****m 17

Issues and Pull Requests

Last synced: about 2 years ago

All Time
  • Total issues: 0
  • Total pull requests: 0
  • Average time to close issues: N/A
  • Average time to close pull requests: N/A
  • Total issue authors: 0
  • Total pull request authors: 0
  • Average comments per issue: 0
  • Average comments per pull request: 0
  • Merged pull requests: 0
  • Bot issues: 0
  • Bot pull requests: 0
Past Year
  • Issues: 0
  • Pull requests: 0
  • Average time to close issues: N/A
  • Average time to close pull requests: N/A
  • Issue authors: 0
  • Pull request authors: 0
  • Average comments per issue: 0
  • Average comments per pull request: 0
  • Merged pull requests: 0
  • Bot issues: 0
  • Bot pull requests: 0
View more stats
Top Authors
Issue Authors
Pull Request Authors
Top Labels
Issue Labels
Pull Request Labels

Dependencies

.github/workflows/push.yml actions
  • actions/checkout v3 composite
  • actions/setup-python v3 composite
.github/workflows/release.yml actions
  • actions/checkout v3 composite
  • actions/setup-python v3 composite
  • pypa/gh-action-pypi-publish master composite
pyproject.toml pypi
  • gymnasium ^0.28
  • matplotlib ^3.7
  • numpy ^1.24
  • pybullet ^3.2.5
  • pytest ^7.3
  • python ^3.10
  • scipy ^1.10
  • stable-baselines3 ^2.0.0
  • transforms3d ^0.4.1