zipfian_environments

This repository contains the environments released as part of the paper "Zipfian Environments for Reinforcement Learning" (https://arxiv.org/abs/2203.08222). It contains subfolders for each of the three environments proposed in the paper: 1) Zipf's Playroom 2) Zipf's Labyrinth 3) Zipf's Gridworld

Installation

zipfs_playroom

These tasks are Unity-based, and are provided through pre-packaged Docker containers.

This package consists of support code to run these Docker containers. You interact with the task environment via a dm_env Python interface.

Requirements

This environment requires Docker, Python 3.6.1 or later and a x86-64 CPU with SSE4.2 support. We do not attempt to maintain a working version for Python 2.

Note: We recommend using Python virtual environment to mitigate conflicts with your system's Python environment.

Download and install Docker:

• For Linux, install Docker-CE
• Install Docker Desktop for OSX or Windows.

Installation

To install zipfs_playroom:

shell python3 -m venv zipfs_playroom_venv source zipfs_playroom_venv/bin/activate pip install --upgrade pip pip install -r ./zipfian_environments/playroom/requirements.txt

zipfs_labyrinth

Zipf's Labyrinth depends on DeepMind Labyrinth, which you will need to install by following the instructions here.

Once that is done, you can install the python requirements by running the following commands:

shell python3 -m venv zipfs_labyrinth_venv source zipfs_labyrinth_venv/bin/activate pip install --upgrade pip pip install -r ./zipfian_environments/labyrinth/requirements.txt

zipfs_gridworld

To install the necessary requirements for Zipf's gridword, you can run the following commands:

shell python3 -m venv zipfs_gridworld_venv source zipfs_gridworld_venv/bin/activate pip install --upgrade pip pip install -r ./zipfian_environments/gridworld/requirements.txt

Usage

All examples assume execution from the directory that you cloned the repository into.

Zipf's Playroom

As an example, you can instantiate a dm_env instance by running the following:

```python import zipfianenvironments.playroom as zipfsplayroom

settings = zipfsplayroom.EnvironmentSettings(seed=123, levelname='lift/liftshapezipf2') env = zipfsplayroom.loadfrom_docker(settings) timestep = env.reset() ```

Zipf's Labyrinth

As an example, you can run the following code to load a Zipf's Labyrinth environment.

``` from zipfianenvironments.labyrinth import zipfslabyrinth

env = zipfslabyrinth.ZipfsLabyrinth( distribution="zipf1", reverseorder=False ) timestep = env.reset() ```

You may first need to set the deepmind lab runfiles path as follows:

``` import deepmind_lab

deepmindlab.setrunfiles_path("INSERT PATH HERE") ```

Zipf's Gridworld

As an example, you can run the following code to load a Zipf's Gridworld environment and test out all the possible actions (note that the code will need to be run from within this directory).

``` from zipfianenvironments.gridworld import zipfsgridworld

env = zipfsgridworld.simplebuilder(levelname='zipf2') timestep = env.reset() for action in range(8): timestep = env.step(action) ```

Citing this work

If you use this work, please cite the following paper @misc{chan2022zipfian, doi = {10.48550/ARXIV.2203.08222}, url = {https://arxiv.org/abs/2203.08222}, author = {Chan, Stephanie C. Y. and Lampinen, Andrew K. and Richemond, Pierre H. and Hill, Felix}, keywords = {Machine Learning (cs.LG), FOS: Computer and information sciences, FOS: Computer and information sciences}, title = {Zipfian environments for Reinforcement Learning}, publisher = {arXiv}, year = {2022}, copyright = {Creative Commons Attribution 4.0 International} }

Disclaimer

This is not an official Google product.