dm-meltingpot

A suite of test scenarios for multi-agent reinforcement learning.

https://github.com/google-deepmind/meltingpot

Science Score: 77.0%

This score indicates how likely this project is to be science-related based on various indicators:

✓
CITATION.cff file
Found CITATION.cff file
✓
codemeta.json file
Found codemeta.json file
✓
.zenodo.json file
Found .zenodo.json file
✓
DOI references
Found 3 DOI reference(s) in README
✓
Academic publication links
Links to: arxiv.org
✓
Committers with academic emails
1 of 14 committers (7.1%) from academic institutions
○
Institutional organization owner
○
JOSS paper metadata
○
Scientific vocabulary similarity
Low similarity (13.4%) to scientific vocabulary

Keywords

multiagent-reinforcement-learning

Keywords from Contributors

distribution interactive mesh interpretability gym-environment sequences generic projection optim hacking

Last synced: 6 months ago · JSON representation ·

Repository

A suite of test scenarios for multi-agent reinforcement learning.

Basic Info

Host: GitHub
Owner: google-deepmind
License: apache-2.0
Language: Python
Default Branch: main
Homepage:
Size: 202 MB

Statistics

Stars: 729
Watchers: 14
Forks: 139
Open Issues: 11
Releases: 15

Topics

multiagent-reinforcement-learning

Created over 4 years ago · Last pushed 7 months ago

Metadata Files

Readme Changelog Contributing License Citation Security Authors

Melting Pot

A suite of test scenarios for multi-agent reinforcement learning.

Melting Pot 2.0 Tech Report Melting Pot Contest at NeurIPS 2023

About

Melting Pot assesses generalization to novel social situations involving both familiar and unfamiliar individuals, and has been designed to test a broad range of social interactions such as: cooperation, competition, deception, reciprocation, trust, stubbornness and so on. Melting Pot offers researchers a set of over 50 multi-agent reinforcement learning substrates (multi-agent games) on which to train agents, and over 256 unique test scenarios on which to evaluate these trained agents. The performance of agents on these held-out test scenarios quantifies whether agents:

perform well across a range of social situations where individuals are interdependent,
interact effectively with unfamiliar individuals not seen during training

The resulting score can then be used to rank different multi-agent RL algorithms by their ability to generalize to novel social situations.

We hope Melting Pot will become a standard benchmark for multi-agent reinforcement learning. We plan to maintain it, and will be extending it in the coming years to cover more social interactions and generalization scenarios.

If you are interested in extending Melting Pot, please refer to the Extending Melting Pot documentation.

Installation

`pip` install

Melting Pot is available on PyPI and can be installed using:

shell pip install dm-meltingpot

NOTE: Melting Pot is built on top of DeepMind Lab2D which is distributed as pre-built wheels. If there is no appropriate wheel for dmlab2d, you will need to build it from source (see the dmlab2d README.md for details).

Manual install

If you want to work on the Melting Pot source code, you can perform an editable installation as follows:

Clone Melting Pot:

shell git clone -b main https://github.com/google-deepmind/meltingpot cd meltingpot
(Optional) Activate a virtual environment, e.g.:

shell python -m venv venv source venv/bin/activate
Install Melting Pot:

shell pip install --editable .[dev]
(Optional) Test the installation:

shell pytest --pyargs meltingpot

Devcontainer (x86 only)

NOTE: This Devcontainer only works for x86 platforms. For arm64 (newer M1 Macs) users will have to follow the manual installation steps.

This project includes a pre-configured development environment (devcontainer).

You can launch a working development environment with one click, using e.g. Github Codespaces or the VSCode Containers extension.

CUDA support

To enable CUDA support (required for GPU training), make sure you have the nvidia-container-toolkit package installed, and then run Docker with the ---gpus all flag enabled. Note that for GitHub Codespaces this isn't necessary, as it's done for you automatically.

Example usage

Evaluation

The evaluation library can be used to evaluate SavedModels trained on Melting Pot substrates.

Evaluation results from the Melting Pot 2.0 Tech Report can be viewed in the Evaluation Notebook.

Interacting with the substrates

You can try out the substrates interactively with the human_players scripts. For example, to play the clean_up substrate, you can run:

shell python meltingpot/human_players/play_clean_up.py

You can move around with the W, A, S, D keys, Turn with Q, and E, fire the zapper with 1, and fire the cleaning beam with 2. You can switch between players with TAB. There are other substrates available in the human_players directory. Some have multiple variants, which you select with the --level_name flag.

Training agents

We provide an illustrative example script using RLlib. However, note that Melting Pot is agnostic to how you train your agents, and this script is not meant to be a suggestion for how to achieve a good score in the task suite. The authors of the suite never used this example training script in their own work.

RLlib

This example uses RLlib to train agents in self-play on a Melting Pot substrate.

First you will need to install the dependencies needed by the examples:

shell cd <meltingpot_root> pip install -r examples/requirements.txt

Then you can run the training experiment using:

shell cd examples/rllib python self_play_train.py

Documentation

Full documentation is available here.

Citing Melting Pot

If you use Melting Pot in your work, please cite the accompanying articles:

bibtex @inproceedings{leibo2021meltingpot, title={Scalable Evaluation of Multi-Agent Reinforcement Learning with Melting Pot}, author={Joel Z. Leibo AND Edgar Du\'e\~nez-Guzm\'an AND Alexander Sasha Vezhnevets AND John P. Agapiou AND Peter Sunehag AND Raphael Koster AND Jayd Matyas AND Charles Beattie AND Igor Mordatch AND Thore Graepel}, year={2021}, journal={International conference on machine learning}, organization={PMLR}, url={https://doi.org/10.48550/arXiv.2107.06857}, doi={10.48550/arXiv.2107.06857} }

bibtex @article{agapiou2022melting, title={Melting Pot 2.0}, author={Agapiou, John P and Vezhnevets, Alexander Sasha and Du{\'e}{\~n}ez-Guzm{\'a}n, Edgar A and Matyas, Jayd and Mao, Yiran and Sunehag, Peter and K{\"o}ster, Raphael and Madhushani, Udari and Kopparapu, Kavya and Comanescu, Ramona and Strouse, {DJ} and Johanson, Michael B and Singh, Sukhdeep and Haas, Julia and Mordatch, Igor and Mobbs, Dean and Leibo, Joel Z}, journal={arXiv preprint arXiv:2211.13746}, year={2022} }

Disclaimer

This is not an officially supported Google product.

Owner

Name: Google DeepMind
Login: google-deepmind
Kind: organization

Website: https://www.deepmind.com/
Repositories: 245
Profile: https://github.com/google-deepmind

Citation (CITATION.bib)

@inproceedings{leibo2021meltingpot,
    title={Scalable Evaluation of Multi-Agent Reinforcement Learning with
           Melting Pot},
    author={Joel Z. Leibo AND Edgar Du\'e\~nez-Guzm\'an AND Alexander Sasha
            Vezhnevets AND John P. Agapiou AND Peter Sunehag AND Raphael Koster
            AND Jayd Matyas AND Charles Beattie AND Igor Mordatch AND Thore
            Graepel},
    year={2021},
    journal={International Conference on Machine Learning},
    organization={PMLR},
    url={https://doi.org/10.48550/arXiv.2107.06857},
    doi={10.48550/arXiv.2107.06857}
}

GitHub Events

Total

Create event: 30
Release event: 1
Issues event: 6
Watch event: 117
Delete event: 23
Issue comment event: 28
Push event: 39
Pull request review event: 11
Pull request event: 58
Fork event: 21

Last Year

Create event: 30
Release event: 1
Issues event: 6
Watch event: 117
Delete event: 23
Issue comment event: 28
Push event: 39
Pull request review event: 11
Pull request event: 58
Fork event: 21

Committers

Last synced: 9 months ago

All Time

Total Commits: 510
Total Committers: 14
Avg Commits per committer: 36.429
Development Distribution Score (DDS): 0.371

Past Year

Commits: 64
Committers: 4
Avg Commits per committer: 16.0
Development Distribution Score (DDS): 0.422

Top Committers

Name	Email	Commits
John Agapiou	j**u@g**m	321
dependabot[bot]	4****]	77
Edgar Duéñez-Guzmán	d**z@g**m	54
Joel Z. Leibo	j**l@g**m	32
Melting Pot Contributor	m**b@g**m	9
Muff2n	r**5@g**m	5
Alan	4****y	5
elliottower	e**t@e**m	1
Shaobo Hou	s**u@g**m	1
Rohan138	r**r@p**u	1
Ninell Oldenburg	5****g	1
Jaime	7****a	1
Charlie Beattie	c**e@g**m	1
dimonenka	d**a@m**u	1

Committer Domains (Top 20 + Academic)

google.com: 6 mail.ru: 1 purdue.edu: 1 elliottower.com: 1

Issues and Pull Requests

Last synced: 6 months ago

All Time

Total issues: 61
Total pull requests: 213
Average time to close issues: about 1 month
Average time to close pull requests: 15 days
Total issue authors: 36
Total pull request authors: 11
Average comments per issue: 3.39
Average comments per pull request: 0.86
Merged pull requests: 78
Bot issues: 0
Bot pull requests: 198

Past Year

Issues: 5
Pull requests: 48
Average time to close issues: 6 months
Average time to close pull requests: 7 days
Issue authors: 5
Pull request authors: 2
Average comments per issue: 1.8
Average comments per pull request: 0.48
Merged pull requests: 21
Bot issues: 0
Bot pull requests: 46

View more stats

Top Authors

Issue Authors

jagapiou (5)
AsadJeewa (5)
ninell-oldenburg (5)
elliottower (4)
neuronphysics (2)
GoingMyWay (2)
Muff2n (2)
mgerstgrasser (2)
s-a-barnett (2)
olipinski (2)
alexunderch (2)
theo-michel (2)
nkenschaft (1)
pseudo-rnd-thoughts (1)
ZzzihaoGuo (1)

Pull Request Authors

dependabot[bot] (218)
Zhihan-Wang-UT (2)
ezhang7423 (2)
imaitland (2)
ninell-oldenburg (2)
elliottower (2)
dimonenka (1)
Viswesh-N (1)
jzleibo (1)
RuizSerra (1)
jagapiou (1)

Top Labels

Issue Labels

bug (4) enhancement (2)

Pull Request Labels

dependencies (217) github_actions (111) python (75) docker (31)

Packages

Total packages: 2
Total downloads:
- pypi 1,283 last-month

Total dependent packages: 4
(may contain duplicates)
Total dependent repositories: 0
(may contain duplicates)
Total versions: 29
Total maintainers: 2

proxy.golang.org: github.com/google-deepmind/meltingpot

Documentation: https://pkg.go.dev/github.com/google-deepmind/meltingpot#section-documentation
License: apache-2.0
Latest release: v2.4.0+incompatible
published about 1 year ago

Versions: 14
Dependent Packages: 0
Dependent Repositories: 0

Rankings

Dependent packages count: 5.4%

Average: 5.6%

Dependent repos count: 5.8%

Last synced: 6 months ago

pypi.org: dm-meltingpot

A suite of test scenarios for multi-agent reinforcement learning.

Homepage: https://github.com/google-deepmind/meltingpot
Documentation: https://dm-meltingpot.readthedocs.io/
License: Apache 2.0
Latest release: 2.4.0
published about 1 year ago

Versions: 15
Dependent Packages: 4
Dependent Repositories: 0
Downloads: 1,283 Last month

Rankings

Stargazers count: 3.0%

Forks count: 5.1%

Dependent packages count: 7.2%

Average: 17.0%

Downloads: 28.4%

Dependent repos count: 41.2%

Maintainers (2)

deepmind jagapiou

Last synced: 6 months ago

Dependencies

.github/workflows/codeql-analysis.yml actions

actions/checkout ac593985615ec2ede58e132d2e21d2b1cbd6127c composite
github/codeql-action/analyze 3ebbd71c74ef574dbc558c82f70e52732c8b44fe composite
github/codeql-action/autobuild 3ebbd71c74ef574dbc558c82f70e52732c8b44fe composite
github/codeql-action/init 3ebbd71c74ef574dbc558c82f70e52732c8b44fe composite

.github/workflows/scorecards-analysis.yml actions

actions/checkout ac593985615ec2ede58e132d2e21d2b1cbd6127c composite
actions/upload-artifact 0b7f8abb1508181956e8e162db84b466c27e18ce composite
github/codeql-action/upload-sarif 3ebbd71c74ef574dbc558c82f70e52732c8b44fe composite
ossf/scorecard-action e38b1902ae4f44df626f11ba0734b14fb91f8f86 composite

.devcontainer/Dockerfile docker

nvidia/cuda 11.8.0-cudnn8-devel-ubuntu20.04@sha256 build

setup.py pypi

absl-py *
chex *
dm_env *
dmlab2d *
immutabledict *
ml-collections *
networkx *
numpy *
pygame *
rx *
tensorflow-macos *

.github/actions/install-examples/action.yml actions

./.github/actions/install-meltingpot * composite

.github/actions/install-meltingpot/action.yml actions

actions/cache/restore v3 composite
actions/cache/save v3 composite
actions/setup-python 61a6322f88396a6271a6ee3565807d608ecaddd1 composite

.github/workflows/pylint-examples.yml actions

./.github/actions/install-examples * composite
actions/checkout f43a0e5ff2bd294095638e18286ca9a3d1956744 composite

.github/workflows/pylint-meltingpot.yml actions

./.github/actions/install-meltingpot * composite
actions/checkout f43a0e5ff2bd294095638e18286ca9a3d1956744 composite

.github/workflows/pypi-publish.yml actions

actions/checkout f43a0e5ff2bd294095638e18286ca9a3d1956744 composite
actions/setup-python 61a6322f88396a6271a6ee3565807d608ecaddd1 composite
pypa/gh-action-pypi-publish b7f401de30cb6434a1e19f805ff006643653240e composite

.github/workflows/pypi-test.yml actions

actions/setup-python 61a6322f88396a6271a6ee3565807d608ecaddd1 composite

.github/workflows/pytype-examples.yml actions

./.github/actions/install-examples * composite
actions/checkout f43a0e5ff2bd294095638e18286ca9a3d1956744 composite

.github/workflows/pytype-meltingpot.yml actions

./.github/actions/install-meltingpot * composite
actions/checkout f43a0e5ff2bd294095638e18286ca9a3d1956744 composite

.github/workflows/test-examples.yml actions

./.github/actions/install-examples * composite
actions/checkout f43a0e5ff2bd294095638e18286ca9a3d1956744 composite

.github/workflows/test-meltingpot.yml actions

./.github/actions/install-meltingpot * composite
actions/checkout f43a0e5ff2bd294095638e18286ca9a3d1956744 composite

examples/requirements.txt pypi

absl_py *
dm_env *
dm_meltingpot *
dm_tree *
dmlab2d *
gymnasium *
matplotlib *
ml_collections *
numpy *
pettingzoo >=1.22.3
ray ==2.5.0
stable_baselines3 *
supersuit >=3.7.2
torch *

pyproject.toml pypi