Data

Tools

Indexes

Applications

Experiments

Open Source Science

Recent Releases of benchmarl

benchmarl - 1.5.0

Discrete MASAC rewrite, static seeding in evaluation, experiment relaoding, tutorials, task restyle and much more

BenchMARL release paired with TorchRL 0.8

BC-Breaking changes: - Decouple Task enums from task classes by @matteobettini in https://github.com/facebookresearch/BenchMARL/pull/175 -> before, tasks were an enum that also contained the logic to create an environment, now the two concepts are separate. Task just contains the enum of available tasks and TaskClass contains the environment creation logic. This has been made to address https://github.com/facebookresearch/BenchMARL/issues/173

[Tutorial] Creating a VMAS scenario and training it https://github.com/facebookresearch/BenchMARL/pull/172

Open In Colab   Creating a VMAS scenario and training it in BenchMARL. We will create a scenario where multiple robots with different embodiments need to navigate to their goals while avoiding each other (as well as obstacles) and train it using MAPPO and MLP/GNN policies.

[Feature] Reload experiment without re-passing config https://github.com/facebookresearch/BenchMARL/pull/179

You can now reload the experiment in many ways just by passing the checkpoint file https://benchmarl.readthedocs.io/en/latest/concepts/features.html#reloading

[Feature, BC-breaking] Decouple Task enums from task classes https://github.com/facebookresearch/BenchMARL/pull/175

before, tasks were an enum that also contained the logic to create an environment, now the two concepts are separate. Task just contains the enum of available tasks and TaskClass contains the environment creation logic. This has been made to address https://github.com/facebookresearch/BenchMARL/issues/173

Example:

python VmasTask.BALANCE # Task VmasTask.BALANCE.get_from_yaml(path=None) # TaskClass, you can pass a custom yaml file path VmasTask.BALANCE.get_task(config=None) # TaskClass, you can pass a custom config if you don't want to load from yaml assert VmasTask.BALANCE.get_from_yaml() == VmasTask.BALANCE.get_task(config=None)

[Feature] Static evaluation https://github.com/facebookresearch/BenchMARL/pull/185

By setting experiment.evaluation_static=True evaluation environments will be the same throughout training and in between runs

[Feature] Discrete MASAC rewrite https://github.com/facebookresearch/BenchMARL/pull/192

https://github.com/facebookresearch/BenchMARL/pull/192 has a really in depth explanation.

TL;DR look how good are the new Discrete MASAC implementations wrt to the old one on VMAS balance

W B Chart 15_04_2025, 10_19_47

What's Changed

  • [Tutorial] Creating a VMAS scenario and training it by @matteobettini in https://github.com/facebookresearch/BenchMARL/pull/172
  • [Feature] Usability improvement in ExperimentConfig by @itwasabhi in https://github.com/facebookresearch/BenchMARL/pull/174
  • Ensure experiment uses user configured buffer device for replay buffer by @itwasabhi in https://github.com/facebookresearch/BenchMARL/pull/180
  • [Feature] Reload experiment without re-passing config by @matteobettini in https://github.com/facebookresearch/BenchMARL/pull/179
  • [Minor] Seeding and logging by @matteobettini in https://github.com/facebookresearch/BenchMARL/pull/183
  • [Docs] Section on rendering and devices by @matteobettini in https://github.com/facebookresearch/BenchMARL/pull/184
  • [Feature, BC-breaking] Decouple Task enums from task classes by @matteobettini in https://github.com/facebookresearch/BenchMARL/pull/175
  • [Feature] Static evaluation by @matteobettini in https://github.com/facebookresearch/BenchMARL/pull/185
  • [Feature] Pass devices to exploration modules by @matteobettini in https://github.com/facebookresearch/BenchMARL/pull/191
  • [Feature] Discrete MASAC by @matteobettini in https://github.com/facebookresearch/BenchMARL/pull/192
  • [Minor] Add kwargs to Plotting methods by @Xmaster6y in https://github.com/facebookresearch/BenchMARL/pull/195
  • [Fine-tune] Restore VMAS gamma by @matteobettini in https://github.com/facebookresearch/BenchMARL/pull/198

New Contributors

  • @itwasabhi made their first contribution in https://github.com/facebookresearch/BenchMARL/pull/174
  • @Xmaster6y made their first contribution in https://github.com/facebookresearch/BenchMARL/pull/195

Full Changelog: https://github.com/facebookresearch/BenchMARL/compare/1.4.0...1.5.0

- Python
Published by matteobettini about 1 year ago

benchmarl - 1.4.0

BenchMARL release paired with TorchRL 0.7.0

A lot of new features and many bug fixes!

New environment: MAgent2

Thans a lot to @JoseLuisC99 for making their first contribution and helping in this effort

  • [Environment] MAgent2 by @JoseLuisC99 in https://github.com/facebookresearch/BenchMARL/pull/137

Parallel collection for non-vectorized envs

Thanks to a lot of help from @gliese876b we have introduced parallel collection for non-vectorized envs using torchrl ParallelEnv

Check it out here:

  • [Feature] Parallel collection by @matteobettini in https://github.com/facebookresearch/BenchMARL/pull/152

Prioritized replay buffers

Again, thanks a mil to @gliese876b

  • [Feature] Prioritized Replay Buffer by @gliese876b in https://github.com/facebookresearch/BenchMARL/pull/160

Ensemble models and algorithms

From the amazing request of @karthiks1701 we have introduced the possibilty of using different models and algorithms for different agent groups

Check it out here: https://benchmarl.readthedocs.io/en/latest/concepts/features.html#ensemble-models-and-algorithms

  • [Feature] Ensemble models and algorithms (different chioces for different agent groups) by @matteobettini in https://github.com/facebookresearch/BenchMARL/pull/159

Disk sotrage for replay buffers

Again, thanks to the input of @JoseLuisC99, we have added the possibility to store buffers on diisk

  • [Feature] Disk storage for replay buffers by @JoseLuisC99 in https://github.com/facebookresearch/BenchMARL/pull/155

VMAS football

Enjoy the full power of the new VMAS football enviornment in BenchMARL

More info here: https://github.com/proroklab/VectorizedMultiAgentSimulator/releases/tag/1.5.0

football_vmas-ezgif com-optimize

What's Changed

  • [Errors] Error on unavailable combinations by @matteobettini in https://github.com/facebookresearch/BenchMARL/pull/142
  • [BugFix] Fix meltingpot buffer transforms by @matteobettini in https://github.com/facebookresearch/BenchMARL/pull/148
  • [BugFix] VMAS parameters names by @matteobettini in https://github.com/facebookresearch/BenchMARL/pull/149
  • [Environment] MAgent2 by @JoseLuisC99 in https://github.com/facebookresearch/BenchMARL/pull/137
  • [Docs] MAgent by @matteobettini in https://github.com/facebookresearch/BenchMARL/pull/150
  • [Minor] Dependencies by @matteobettini in https://github.com/facebookresearch/BenchMARL/pull/151
  • [Feature] Parallel collection by @matteobettini in https://github.com/facebookresearch/BenchMARL/pull/152
  • [Feature] Prioritized Replay Buffer by @gliese876b in https://github.com/facebookresearch/BenchMARL/pull/160
  • [Docs] MAgent typo by @matteobettini in https://github.com/facebookresearch/BenchMARL/pull/162
  • [Feature] Ensemble models and algorithms (different chioces for different agent groups) by @matteobettini in https://github.com/facebookresearch/BenchMARL/pull/159
  • [Feature] Disk storage for replay buffers by @JoseLuisC99 in https://github.com/facebookresearch/BenchMARL/pull/155
  • [BugFix] Tensorboard hparam logging by @matteobettini in https://github.com/facebookresearch/BenchMARL/pull/165
  • [TorchRL deprecation] AdditiveGaussianWrapper -> AdditiveGaussianModule by @matteobettini in https://github.com/facebookresearch/BenchMARL/pull/167
  • [Task] VMAS football by @matteobettini in https://github.com/facebookresearch/BenchMARL/pull/166
  • [Fine-tune] Update gammas by @matteobettini in https://github.com/facebookresearch/BenchMARL/pull/168
  • [Dependency] Make video rendering deps core by @matteobettini in https://github.com/facebookresearch/BenchMARL/pull/169
  • [Release] 1.4.0 by @matteobettini in https://github.com/facebookresearch/BenchMARL/pull/170

New Contributors

  • @JoseLuisC99 made their first contribution in https://github.com/facebookresearch/BenchMARL/pull/137
  • @gliese876b made their first contribution in https://github.com/facebookresearch/BenchMARL/pull/160

Full Changelog: https://github.com/facebookresearch/BenchMARL/compare/1.3.0...1.4.0

- Python
Published by matteobettini over 1 year ago

benchmarl - 1.3.0

Memory networks in BenchMARL

BenchMARL release paired with TorchRL 0.6.0

Highlights

RNNs in BenchMARL

We now support RNNs as models!

We have implemented and added to the library GRU and LSTM.

These can be used in the policy or in the critic (both from local agent inputs and from global centralized inputs). They are also compatible with any parameter sharing choice.

We have benchmarked GRU on a multiagent version of repeat_previous and it does solve the task, while just MLP fails

W B Chart 01_08_2024, 09_17_59

In cotrast to traditional RNNs, we do not do any time padding. Instead we do a for loop over time in the sequence while reading the “is_init” flag. This approach is slower but leads to better results.

Oh also as always you can chain them with as many models as you desire (CNN, GNN, ...)

Simplified model reloading and evaluation

We have added some useful tools described at here

In particular, we have added experiment.evaluate() and some useful command line tols like benchmarl/evaluate.py and benchmarl/resume.py that just take the path to a checkpoint file.

Basically now you can reload models from the hydra run without giving again all the config, the scripts will find the configs you used in the hydra folders automatically.

Better logging of episode metrics

BenchaMARL will now consider an episode done when the global task done is set. Thus, it will allow for agents done early (as long as the global done is set on all()

Here is an overview of how episode metrics are computed:

epispde_reward compuation

BenchMARL will be looking at the global done (always assumed to be set), which can usually be computed using any or all over the single agents dones.

In all cases the global done is what is used to compute the episode reward.

We log episode_reward min, mean, max over episodes at three different levels: - agent, (disabled by default, can be turned on manually) - group, averaged over agents in group - global, averaged over agents in groups and gropus

What's Changed

  • [BugFix] Fix collect with grad by @matteobettini in https://github.com/facebookresearch/BenchMARL/pull/114
  • [Refactor] Update values headed to deprecation by @matteobettini in https://github.com/facebookresearch/BenchMARL/pull/118
  • [Docs] Citation by @matteobettini in https://github.com/facebookresearch/BenchMARL/pull/119
  • [Model] GRU and general RNN compatibility by @matteobettini in https://github.com/facebookresearch/BenchMARL/pull/116
  • [Model] LSTM by @matteobettini in https://github.com/facebookresearch/BenchMARL/pull/120
  • [BugFix, Feature] GNN position_key and velocity_key not in observation_spec by @matteobettini in https://github.com/facebookresearch/BenchMARL/pull/125
  • [BugFix] Small fix to multi-group eval and add wandb project_name by @matteobettini in https://github.com/facebookresearch/BenchMARL/pull/126
  • [Feature] Improved experiment reloading and evaluation by @matteobettini in https://github.com/facebookresearch/BenchMARL/pull/127
  • [BugFix] GNN position and velocity key by @matteobettini in https://github.com/facebookresearch/BenchMARL/pull/132
  • [BugFix] More flexible episode_reward computation in logger by @matteobettini in https://github.com/facebookresearch/BenchMARL/pull/136
  • [Feature] RNNs in SAC losses by @matteobettini in https://github.com/facebookresearch/BenchMARL/pull/139
  • [Version] 1.3.0 by @matteobettini in https://github.com/facebookresearch/BenchMARL/pull/140

Full Changelog: https://github.com/facebookresearch/BenchMARL/compare/1.2.1...1.3.0

- Python
Published by matteobettini over 1 year ago

benchmarl - 1.2.1

BenchMARL release paired with TorchRL 0.4.0

Many exctiting updates in this new release!

Most importantly there have been major inmpovements to the GNN and the addition of a new model: DeepSets

1718290825438

New features

  • [Feature] Allow multiple observation keys in specs by @matteobettini in https://github.com/facebookresearch/BenchMARL/pull/82 https://github.com/facebookresearch/BenchMARL/pull/83
  • [Feature] Buffer device by @matteobettini in https://github.com/facebookresearch/BenchMARL/pull/87
  • [Feature] More VMAS tasks by @matteobettini in https://github.com/facebookresearch/BenchMARL/pull/88
  • [Feature] Simplify extending benchmarl and improve extending examples by @matteobettini in https://github.com/facebookresearch/BenchMARL/pull/89
  • [Example] Collecting with gradient by @matteobettini in https://github.com/facebookresearch/BenchMARL/pull/77
  • [Feature] Share parameters between models by @matteobettini in https://github.com/facebookresearch/BenchMARL/pull/95
  • [Feature] GNN build topology dynamically from positions by @matteobettini in https://github.com/facebookresearch/BenchMARL/pull/98
  • [Model] DeepSets by @matteobettini in https://github.com/facebookresearch/BenchMARL/pull/96
  • [Feature] PPO minibatch advantage by @matteobettini in https://github.com/facebookresearch/BenchMARL/pull/100
  • [Feature] keep_checkpoints_num and checkpoint_at_end by @matteobettini in https://github.com/facebookresearch/BenchMARL/pull/102
  • [Feature] Run evaluation on first iteration by @matteobettini in https://github.com/facebookresearch/BenchMARL/pull/108
  • [Feature] Iteration timer includes evaluation by @matteobettini in https://github.com/facebookresearch/BenchMARL/pull/110

New Contributors

  • @mchoilab made their first contribution in https://github.com/facebookresearch/BenchMARL/pull/105

Full Changelog: https://github.com/facebookresearch/BenchMARL/compare/1.2.0...1.2.1

- Python
Published by matteobettini almost 2 years ago

benchmarl - 1.2.0

BenchMARL release paired with TorchRL 0.4.0

BenchMARL just got extended with 49 vision-based tasks from Google DeepMind MeltingPot 🎉

But wait there is more! We have added a brand new CNN model 🚀 that can be used in actors and critics (fully compatible with any parameter sharing and centralization choices)

What's Changed

  • [Feature] Allow any name for observation and global_state keys by @matteobettini in https://github.com/facebookresearch/BenchMARL/pull/72
  • [Model] CNN by @matteobettini in https://github.com/facebookresearch/BenchMARL/pull/74
  • [Feature] Allow multipe inputs to models by @matteobettini in https://github.com/facebookresearch/BenchMARL/pull/73
  • [Environment] Melitngpot by @matteobettini in https://github.com/facebookresearch/BenchMARL/pull/75

Full Changelog: https://github.com/facebookresearch/BenchMARL/compare/1.1.1...1.2.0

- Python
Published by matteobettini about 2 years ago

benchmarl - 1.1.1

Align to torchrl 0.3.1

What's Changed

  • [CI] Fix CI by @matteobettini in https://github.com/facebookresearch/BenchMARL/pull/65
  • [Test] Lighter tests by @matteobettini in https://github.com/facebookresearch/BenchMARL/pull/66
  • [CI] Fix CI by @matteobettini in https://github.com/facebookresearch/BenchMARL/pull/67
  • [Release] 1.1.1 by @matteobettini in https://github.com/facebookresearch/BenchMARL/pull/69

Full Changelog: https://github.com/facebookresearch/BenchMARL/compare/1.1.0...1.1.1

- Python
Published by matteobettini about 2 years ago

benchmarl - 1.1.0

Highlights

What's Changed

  • [Model] GNN by @matteobettini in https://github.com/facebookresearch/BenchMARL/pull/30
  • [Docs] Citation by @matteobettini in https://github.com/facebookresearch/BenchMARL/pull/39
  • [BugFix] Fix discrete SAC by @matteobettini in https://github.com/facebookresearch/BenchMARL/pull/40
  • [Tasks] VMAS MPE (Vectorized) by @matteobettini in https://github.com/facebookresearch/BenchMARL/pull/42
  • [Feature] Optional Tanh transfromed distributions in algorithms by @matteobettini in https://github.com/facebookresearch/BenchMARL/pull/45
  • [Feature] Choose if evaluation is run on sampled actions or on mode by @matteobettini in https://github.com/facebookresearch/BenchMARL/pull/49
  • [BugFix] Use correct types to prevent multiwalker init error. by @KaleabTessera in https://github.com/facebookresearch/BenchMARL/pull/47
  • [CI] Fix code cov by @matteobettini in https://github.com/facebookresearch/BenchMARL/pull/51
  • [Feature] More VMAS tasks by @matteobettini in https://github.com/facebookresearch/BenchMARL/pull/48
  • [Doc] Fix Typo by @matteobettini in https://github.com/facebookresearch/BenchMARL/pull/57
  • [BugFix] Rename PPO params by @matteobettini in https://github.com/facebookresearch/BenchMARL/pull/59
  • [Doc] Fix typo by @matteobettini in https://github.com/facebookresearch/BenchMARL/pull/60
  • [CI] Run main tests with torchrl nightly and have a separate CI for testing torchrl stable by @matteobettini in https://github.com/facebookresearch/BenchMARL/pull/53
  • [Refactor] Adapt PPO parameter names by @matteobettini in https://github.com/facebookresearch/BenchMARL/pull/62
  • [Versioning] 1.1.0 by @matteobettini in https://github.com/facebookresearch/BenchMARL/pull/61

New Contributors

  • @KaleabTessera made their first contribution in https://github.com/facebookresearch/BenchMARL/pull/47

Full Changelog: https://github.com/facebookresearch/BenchMARL/compare/1.0.0...1.1.0

- Python
Published by matteobettini over 2 years ago

benchmarl - 1.0.0

BenchMARL docs

The BenchMARL docs are now available at https://benchmarl.readthedocs.io/.

What's Changed

  • [Feature] More callbacks by @matteobettini in https://github.com/facebookresearch/BenchMARL/pull/35
  • [Docs] Link video presentation by @matteobettini in https://github.com/facebookresearch/BenchMARL/pull/36
  • [BugFix] Task loading schema validation by @matteobettini in https://github.com/facebookresearch/BenchMARL/pull/37
  • [Docs] Build the docs by @matteobettini in https://github.com/facebookresearch/BenchMARL/pull/34

Full Changelog: https://github.com/facebookresearch/BenchMARL/compare/0.0.4...1.0.0

- Python
Published by matteobettini over 2 years ago

benchmarl - 0.0.4

- Python
Published by matteobettini over 2 years ago

benchmarl - 0.0.3

  • Added yaml configuration files to PyPi package
  • Addded possibility to access the experiment from the algorithm

- Python
Published by matteobettini over 2 years ago

benchmarl - 0.0.2

First release

- Python
Published by matteobettini over 2 years ago