Data

Tools

Indexes

Applications

Experiments

Open Source Science

irl-imitation

Implementation of Inverse Reinforcement Learning (IRL) algorithms in Python/Tensorflow. Deep MaxEnt, MaxEnt, LPIRL

https://github.com/yrlu/irl-imitation

Science Score: 77.0%

This score indicates how likely this project is to be science-related based on various indicators:

  • ✓
    CITATION.cff file
    Found CITATION.cff file
  • ✓
    codemeta.json file
    Found codemeta.json file
  • ✓
    .zenodo.json file
    Found .zenodo.json file
  • ✓
    DOI references
    Found 4 DOI reference(s) in README
  • ✓
    Academic publication links
    Links to: arxiv.org, zenodo.org
  • ✓
    Committers with academic emails
    1 of 2 committers (50.0%) from academic institutions
  • ○
    Institutional organization owner
  • ○
    JOSS paper metadata
  • ○
    Scientific vocabulary similarity
    Low similarity (7.5%) to scientific vocabulary

Keywords

imitation imitation-learning inverse-reinforcement-learning irl learning-from-demonstration lfd machine-learning ml reinforcement-learning rl tensorflow
Last synced: 6 months ago · JSON representation ·

Repository

Implementation of Inverse Reinforcement Learning (IRL) algorithms in Python/Tensorflow. Deep MaxEnt, MaxEnt, LPIRL

Basic Info
  • Host: GitHub
  • Owner: yrlu
  • Language: Python
  • Default Branch: master
  • Homepage:
  • Size: 23.1 MB
Statistics
  • Stars: 625
  • Watchers: 30
  • Forks: 150
  • Open Issues: 4
  • Releases: 1
Topics
imitation imitation-learning inverse-reinforcement-learning irl learning-from-demonstration lfd machine-learning ml reinforcement-learning rl tensorflow
Created over 8 years ago · Last pushed almost 2 years ago
Metadata Files
Readme Citation

README.md

irl-imitation

DOI

Implementation of selected Inverse Reinforcement Learning (IRL) algorithms in Python/Tensorflow.

bash $ python demo.py

Implemented Algorithms
  • Linear inverse reinforcement learning (Ng & Russell, 2000)
  • Maximum entropy inverse reinforcement learning (Ziebart et al., 2008)
  • Maximum entropy deep inverse reinforcement learning (Wulfmeier et al., 2015)
Implemented MDPs & Solver
  • 2D gridworld
  • 1D gridworld
  • Value iteration

If you use this software in your publications, please cite it using the following BibTeX entry:

bibtex @misc{lu2017irl-imitation, author = {Lu, Yiren}, doi = {10.5281/zenodo.6796157}, month = {7}, title = {{Implementations of inverse reinforcement learning algorithms in Python/Tensorflow}}, url = {https://github.com/yrlu/irl-imitation}, year = {2017} }

Dependencies

  • python 2.7
  • cvxopt
  • Tensorflow 0.12.1
  • matplotlib

Linear Inverse Reinforcement Learning

bash $ python linear_irl_gridworld.py --act_random=0.3 --gamma=0.5 --l1=10 --r_max=10

Maximum Entropy Inverse Reinforcement Learning

(This implementation is largely influenced by Matthew Alger's maxent implementation)

bash $ python maxent_irl_gridworld.py --height=10 --width=10 --gamma=0.8 --n_trajs=100 --l_traj=50 --no-rand_start --learning_rate=0.01 --n_iters=20

bash $ python maxent_irl_gridworld.py --gamma=0.8 --n_trajs=400 --l_traj=50 --rand_start --learning_rate=0.01 --n_iters=20

Maximum Entropy Deep Inverse Reinforcement Learning

  • Following Wulfmeier et al. 2015 paper: Maximum Entropy Deep Inverse Reinforcement Learning. FC version implemented. The implementation does not follow exactly the model proposed in the paper. Some tweaks applied including elu activations, clipping gradients, l2 regularization etc.
  • $ python deep_maxent_irl_gridworld.py --help for options descriptions

bash $ python deep_maxent_irl_gridworld.py --learning_rate=0.02 --n_trajs=200 --n_iters=20

MIT License

Owner

  • Name: Yiren Lu
  • Login: yrlu
  • Kind: user
  • Location: New York
  • Company: Waymo Research

M.S.E. in Robotics '17, GRASP Lab, UPenn

Citation (CITATION.cff) 

cff-version: 1.2.0
message: "If you use this software, please cite it as below."
authors:
- family-names: "Lu"
  given-names: "Yiren"
  orcid: "https://orcid.org/0000-0002-7924-2488"
title: "yrlu/irl-imitation: Implementation of Inverse Reinforcement Learning (IRL) algorithms in python/Tensorflow"
version: 1.0.0
doi: 10.5281/zenodo.6796157
date-released: 2017-07-01
url: "https://github.com/yrlu/irl-imitation"

GitHub Events

Total
  • Watch event: 56
  • Fork event: 4
Last Year
  • Watch event: 56
  • Fork event: 4

Committers

Last synced: over 2 years ago

All Time
  • Total Commits: 45
  • Total Committers: 2
  • Avg Commits per committer: 22.5
  • Development Distribution Score (DDS): 0.089
Past Year
  • Commits: 0
  • Committers: 0
  • Avg Commits per committer: 0.0
  • Development Distribution Score (DDS): 0.0
Top Committers
Name Email Commits
Max Lu l****n@s****u 41
Yiren Lu l****x@g****m 4
Committer Domains (Top 20 + Academic)
seas.upenn.edu: 1