ReinforcementLearning

ReinforcementLearning: A Package to Perform Model-Free Reinforcement Learning in R - Published in JOSS (2019)

https://github.com/nproellochs/reinforcementlearning

Keywords

Scientific Fields

Repository

This package performs model-free reinforcement learning in R.

README.Rmd

---
output: github_document
---



```{r, echo = FALSE}
knitr::opts_chunk$set(
  collapse = TRUE,
  comment = "#>",
  fig.path = "README-"
)
```

```{r, include=FALSE}
set.seed(0)
```

# Reinforcement Learning

[![Build Status](https://travis-ci.org/nproellochs/ReinforcementLearning.svg?branch=master)](https://travis-ci.org/nproellochs/ReinforcementLearning)
[![CRAN_Status_Badge](http://www.r-pkg.org/badges/version/ReinforcementLearning)](https://cran.r-project.org/package=ReinforcementLearning)
[![DOI](http://joss.theoj.org/papers/10.21105/joss.01087/status.svg)](https://doi.org/10.21105/joss.01087)

**ReinforcementLearning** performs model-free reinforcement learning in R. This implementation enables the learning of
    an optimal policy based on sample sequences consisting of states, actions and rewards. In 
    addition, it supplies multiple predefined reinforcement learning algorithms, such as experience 
    replay.

## Overview

The most important functions of **ReinforcementLearning** are:

- Learning an optimal policy from a fixed set of a priori known transition samples
- Predefined learning rules and action selection modes
- A highly customizable framework for model-free reinforcement learning tasks

## Installation

You can easily install the latest version of **ReinforcementLearning** with

```{r,eval=FALSE}
# Recommended option: download and install latest version from CRAN
install.packages("ReinforcementLearning")

# Alternatively, install the development version from GitHub:
# install.packages("devtools")
devtools::install_github("nproellochs/ReinforcementLearning")
```

## Usage

This section shows the basic functionality of how to perform reinforcement learning. First, load the corresponding package **ReinforcementLearning**.

```{r, message=FALSE}
library(ReinforcementLearning)
```

The following example shows how to learn a reinforcement learning agent using input data in the form of sample sequences 
consisting of states, actions and rewards. The result of the learning process is a state-action table and an optimal 
policy that defines the best possible action in each state. 

```{r, message=FALSE}
# Generate sample experience in the form of state transition tuples
data <- sampleGridSequence(N = 1000)
head(data)

# Define reinforcement learning parameters
control <- list(alpha = 0.1, gamma = 0.1, epsilon = 0.1)

# Perform reinforcement learning
model <- ReinforcementLearning(data, s = "State", a = "Action", r = "Reward", 
                               s_new = "NextState", control = control)

# Print result
print(model)
```

## Learning Reinforcement Learning

If you are new to reinforcement learning, you are better off starting with a systematic introduction, rather than trying to learn from reading individual documentation pages. There are three good places to start:

1.  A thorough introduction to reinforcement learning is provided in [Sutton  (1998)](https://www.semanticscholar.org/paper/Reinforcement-Learning%3A-An-Introduction-Sutton-Barto/dd90dee12840f4e700d8146fb111dbc863a938ad).
    
2.  The package [vignette](https://github.com/nproellochs/ReinforcementLearning/blob/master/vignettes/ReinforcementLearning.Rmd) demonstrates the main functionalities of the ReinforcementLearning R package by drawing upon common examples from the literature (e.g. finding optimal game strategies).

3.  Multiple blog posts on [R-bloggers](https://www.r-bloggers.com/reinforcement-learning-q-learning-with-the-hopping-robot/) demonstrate the capabilities of the ReinforcementLearning package using practical examples.

## Contributing

If you experience any difficulties with the package, or have suggestions, or want to contribute directly, you have the following options:

* Contact the [maintainer](mailto:nicolas.proellochs@wi.jlug.de) by email.
* Issues and bug reports: [File a GitHub issue](https://github.com/nproellochs/ReinforcementLearning/issues).
* Fork the source code, modify, and issue a [pull request](https://help.github.com/articles/creating-a-pull-request-from-a-fork/) through the [project GitHub page](https://github.com/nproellochs/ReinforcementLearning).

## License

**ReinforcementLearning** is released under the [MIT License](https://opensource.org/licenses/MIT)

Copyright (c) 2019 Nicolas Pröllochs & Stefan Feuerriegel

Owner

JOSS Publication

ReinforcementLearning: A Package to Perform Model-Free Reinforcement Learning in R

Published

June 12, 2019

DOI

10.21105/joss.01087

Volume 4, Issue 38, Page 1087

Authors

Nicolas Pröellochs
University of Giessen, University of Oxford

Stefan Feuerriegel
ETH Zurich

Editor

Arfon Smith

Tags

Reinforcement Learning Batch Learning Experience Replay Q-Learning

View PDF Review Thread Software Archive

GitHub Events

Committers

Last synced: 7 months ago

Issues and Pull Requests

Last synced: 6 months ago