vmas - 1.5.0

Football revamp

football_vmas-ezgif com-optimize

We have just revamped VMAS football!

New rewards, observations, options for physical differences and formation spawning and value function plotting.

Try it out for some intense N vs M continuous control gameplay.

We also provide an heuristic opponent with tunable strengths for precision, decision, and speed.

Here are the key new points - New rewards: sparse reward for scoring, dense reward for moving the ball towards the goal, and dense reward for getting at least one agent close to the ball. These rewards can be tuned and removed in a curriculum setting. - Opponent heuristic with tunable strengths: It is possible to use an heuristic as the opponent in order to fairly benchmark different algorithms. The heuristic (referret to as AI in the scenario code) has 3 types of tunable strengths, each ranging from 0 to 1: speed strength, decision strength, and precision strength. Speed strength simply dictates the speed of all movement, from dribbling to off-the-ball movement. A value of 0 yields completely stationary agents, and a value of 1 yields a behavior that constantly operates at the maximum allowable velocity. Decision strength affects the agents’ decision-making by adding noise to the value functions that dictate the agent’s decisions. This affects both movement decisions (i.e. the agent might choose to navigate to a position with a lower value, especially if the values have similar magnitude), as well as possession decisions (i.e. the agents will not be able to accurately evaluate whose possession it is, and might consequently assign the ball to the wrong teammate). Lastly, precision strength controls the agent’s ability to execute planned maneuvers. - Physical differences: you can turn on physical differences to make agents different. Goalkeeper agents will be bigger and slower, while attackers will be smaller and faster - Formation spawaning: You can spawn your agents at random or in programmatic formations - Kiccking and non-kicking agents: By default, agents will just use their 2D movement and collisions to move around and move the ball. By turning on shooting actions, agents will have 2 new further actions for rotating and kicking the ball with a continuous force. (All actions are discretizable as in any VMAS scenario) - Self-play: The setup of observations and rewards is mirrored for the 2 teams. Thus, if instead of using the heuristic opponents you want to use self-play or any other framework that controls both teams you can.

For more technical details on the scenario, see section C.1 of this paper

What's Changed

[BugFix] Windows paths by @matteobettini in https://github.com/proroklab/VectorizedMultiAgentSimulator/pull/148
[Minor] Update links for the CPM road traffic scenario by @Jianye-Xu in https://github.com/proroklab/VectorizedMultiAgentSimulator/pull/150
[Scenario] New football by @matteobettini in https://github.com/proroklab/VectorizedMultiAgentSimulator/pull/97

Full Changelog: https://github.com/proroklab/VectorizedMultiAgentSimulator/compare/1.4.3...1.5.0

- Python
Published by matteobettini over 1 year ago

vmas - 1.4.3

As always, we have a LOT of new updates and this time they are mostly community-driven.

Ovewrview

Terminated/truncated support and Gymnasium wrapper #143

Thanks to @LukasSchaefer, we now have the option to return terminated and truncated from vmas envs instead of done.

We further have wrappers for gymnasium (single vmas env) and vectorized gymnasium, which wraps a batched vmas environemnt, maintaining the n_envs dimension in the data.

Choose how you want to discretize continuous actions #119

Before, continuous actions were discretized in 3 options: increase, decrease, stay

Now, thanks to @rikifunt, you can choose how many options you want to split the continuous range in!

Vectorized lidars #124

Lidars have multiple rays that can see multiple entities. Thanks to @Zartris we know have vectorized both these dimensions making scaling seamless in the number of rays.

VIsualize the boundaries of the world #142

Thanks to @Giovannibriglia, if you set the world bouindaries in a scenario, VMAS now automatically plots them

More dynamics #125

Static: no movement actions
Forward: just froward and backwards force in the direction of the orientation
Rotation: just rotation action

Breaking changes

env.unwrapped() -> env.unwrapped in GymWrapper
removed from "discovery" a redundant agent.state.pos in observation

What's Changed

[Feature] Joint rotation offset and more dynamics by @matteobettini in https://github.com/proroklab/VectorizedMultiAgentSimulator/pull/125
[BugFix] Fixed joints inferred angle by @matteobettini in https://github.com/proroklab/VectorizedMultiAgentSimulator/pull/127
[Feature] Warn if spwaning entities takes too long by @matteobettini in https://github.com/proroklab/VectorizedMultiAgentSimulator/pull/128
[Feature] different number of actions for each action dimension (discrete actions) by @rikifunt in https://github.com/proroklab/VectorizedMultiAgentSimulator/pull/119
Introduce xsemidim and ysemidim parameters to enable customizable environment dimensions by @Giovannibriglia in https://github.com/proroklab/VectorizedMultiAgentSimulator/pull/133
[BugFix] Discovery obs by @matteobettini in https://github.com/proroklab/VectorizedMultiAgentSimulator/pull/137
[Bug fix] Navigation scenario: ensure entity placement within constrained environment boundaries by @Giovannibriglia in https://github.com/proroklab/VectorizedMultiAgentSimulator/pull/139
[Feature] Enable to set the number of lidar rays in navigation, discovery and flocking scenarios by @Giovannibriglia in https://github.com/proroklab/VectorizedMultiAgentSimulator/pull/141
[Feature] Boundary visualization for limited-size environments by @Giovannibriglia in https://github.com/proroklab/VectorizedMultiAgentSimulator/pull/142
[Feature] Terminated/truncated support and Gymnasium wrapper by @LukasSchaefer in https://github.com/proroklab/VectorizedMultiAgentSimulator/pull/143

New Contributors

@rikifunt made their first contribution in https://github.com/proroklab/VectorizedMultiAgentSimulator/pull/119
@Giovannibriglia made their first contribution in https://github.com/proroklab/VectorizedMultiAgentSimulator/pull/133
@LukasSchaefer made their first contribution in https://github.com/proroklab/VectorizedMultiAgentSimulator/pull/143

Full Changelog: https://github.com/proroklab/VectorizedMultiAgentSimulator/compare/1.4.2...1.4.3

- Python
Published by matteobettini over 1 year ago

vmas - 1.4.2

VMAS 1.4.2

This time we have a LOT of exciting updates and community contributions.

New road traffic scenario

First and foremost, we have a new road_traffic scenario thanks to the formidable work of @Jianye-Xu !

Here is a gif to stimulate your appetite

drawing

Joints are more powerfull than ever

In https://github.com/proroklab/VectorizedMultiAgentSimulator/pull/113 we made serious improvements to joints.

Now you can connect n-bodies to form new physical bodies that share all forces and torques.

Check out the renderings in https://github.com/proroklab/VectorizedMultiAgentSimulator/pull/113 for examples of what can be done now in VMAS with joints

Here is a taster

342393854-cc920e53-cb03-4bc4-99ce-0c58e27ef08e-ezgif com-video-to-gif-converter (1)

LIDAR improvements

@Zartris made lidar his mission and helped on a lot of useful LIDAR features

Lidar angles are now relative to the agent rotation https://github.com/proroklab/VectorizedMultiAgentSimulator/pull/101
rendering lidars is now optional https://github.com/proroklab/VectorizedMultiAgentSimulator/pull/108
imporvement to lidar rendering and transparency https://github.com/proroklab/VectorizedMultiAgentSimulator/pull/115

New `pre_step` and `post_step` scenario functions

@Zartris also made creating scenarios way easier with the addition of these 2 functions.

Check them out in https://github.com/proroklab/VectorizedMultiAgentSimulator/pull/112

Kwargs check in scenarios

Thanks to @KaleabTessera we now have better kwargs checking that will warn you if you pass an unexpected parameter to a scenario https://github.com/proroklab/VectorizedMultiAgentSimulator/pull/117

What's Changed

[Rendering] Import matplotplib only if necessary by @matteobettini in https://github.com/proroklab/VectorizedMultiAgentSimulator/pull/96
[Feature] Make lidar angles relative to agent rotation by @matteobettini in https://github.com/proroklab/VectorizedMultiAgentSimulator/pull/101
[BugFix] Fix dynamics by @matteobettini in https://github.com/proroklab/VectorizedMultiAgentSimulator/pull/102
[Tests] MacOS runner to Apple silicon by @matteobettini in https://github.com/proroklab/VectorizedMultiAgentSimulator/pull/103
[CI] FIx CI by @matteobettini in https://github.com/proroklab/VectorizedMultiAgentSimulator/pull/105
[Feature] Optional sensor render by @matteobettini in https://github.com/proroklab/VectorizedMultiAgentSimulator/pull/108
[Feature] Remove extra observation call by @matteobettini in https://github.com/proroklab/VectorizedMultiAgentSimulator/pull/111
[Feature] Adding pre and post step to BaseScenario by @Zartris in https://github.com/proroklab/VectorizedMultiAgentSimulator/pull/112
[Feature] Joint rotations by @matteobettini in https://github.com/proroklab/VectorizedMultiAgentSimulator/pull/113
[Feature] Warn when kwargs passed to a Scenario are not used by @matteobettini in https://github.com/proroklab/VectorizedMultiAgentSimulator/pull/117
[Tests] Improve tests by @matteobettini in https://github.com/proroklab/VectorizedMultiAgentSimulator/pull/118
[Feat] Render lidar before agents and allow for tuple as color by @Zartris in https://github.com/proroklab/VectorizedMultiAgentSimulator/pull/115
[New Scenario] Road Traffic by @Jianye-Xu in https://github.com/proroklab/VectorizedMultiAgentSimulator/pull/99
[Rendering] Add ellipse by @matteobettini in https://github.com/proroklab/VectorizedMultiAgentSimulator/pull/123

New Contributors

@Zartris made their first contribution in https://github.com/proroklab/VectorizedMultiAgentSimulator/pull/112

Full Changelog: https://github.com/proroklab/VectorizedMultiAgentSimulator/compare/1.4.1...1.4.2

- Python
Published by matteobettini almost 2 years ago

vmas - VMAS-1.4.1

Highlights

Drone dynamics by @gy2256 in https://github.com/proroklab/VectorizedMultiAgentSimulator/pull/83 !!!
Fix a bug where the package mass was not passed to the constructor in the transport scenarios by @kfu02 in https://github.com/proroklab/VectorizedMultiAgentSimulator/pull/91
Introduce scenario.render_origin by @matteobettini in https://github.com/proroklab/VectorizedMultiAgentSimulator/pull/93
Compatibility with Python 3.11 by @matteobettini in https://github.com/proroklab/VectorizedMultiAgentSimulator/pull/95
Better tests and CI on all platforms https://github.com/proroklab/VectorizedMultiAgentSimulator/pull/89
Now VMAS has docs!

What's Changed

[Dynamics] Drone dynamics by @gy2256 in https://github.com/proroklab/VectorizedMultiAgentSimulator/pull/83
[CodeQuality] Pre-commit formatting by @matteobettini in https://github.com/proroklab/VectorizedMultiAgentSimulator/pull/87
[Docs] First instance of basic docs by @matteobettini in https://github.com/proroklab/VectorizedMultiAgentSimulator/pull/85
[Docs] Usage by @matteobettini in https://github.com/proroklab/VectorizedMultiAgentSimulator/pull/88
[Test] Better tests and multi OS by @matteobettini in https://github.com/proroklab/VectorizedMultiAgentSimulator/pull/89
[Notbooks] Refresh notebooks by @matteobettini in https://github.com/proroklab/VectorizedMultiAgentSimulator/pull/90
[Bug] Package mass arg now passed into constructor by @kfu02 in https://github.com/proroklab/VectorizedMultiAgentSimulator/pull/91
[Render] Introduce scenario.render_origin by @matteobettini in https://github.com/proroklab/VectorizedMultiAgentSimulator/pull/93
[Quality] Python 3.11 by @matteobettini in https://github.com/proroklab/VectorizedMultiAgentSimulator/pull/95

New Contributors

@kfu02 made their first contribution in https://github.com/proroklab/VectorizedMultiAgentSimulator/pull/91

Full Changelog: https://github.com/proroklab/VectorizedMultiAgentSimulator/compare/VMAS-1.4.0...VMAS-1.4.1

- Python
Published by matteobettini about 2 years ago

vmas - VMAS-1.4.0

Differentiable VMAS

That's right, VMAS is now fully differentiable!

How do I use it?

Just set grad_enabled=True at environement construction time and have any input that requires gradients. This can be actions or scenario parameters. VMAS will keep track of the computation graph on that tensor over time.

What does it mean?

It means that you can differentiate any VMAS output, enabling differentaition of the transition dynamics, reward functions, and observation functions.

Why is it useful?

You can now optimize parameters in VMAS scenarios (e.g., parameters of the various scenario functions or simply initial state values) using losses computed on rewards or observations. It further allows you to backpropagate through time (simulation steps).

What's Changed

[Feature] Differentiable VMAS by @matteobettini in https://github.com/proroklab/VectorizedMultiAgentSimulator/pull/80
[Refactor] DIff drive dynamics by @matteobettini in https://github.com/proroklab/VectorizedMultiAgentSimulator/pull/81

Full Changelog: https://github.com/proroklab/VectorizedMultiAgentSimulator/compare/VMAS-1.3.4...VMAS-1.4.0

- Python
Published by matteobettini over 2 years ago

vmas - VMAS-1.3.4

Highlights

VMAS actions are now decoupled from physics. This allows to define custom agent dynamic models (Holonomic,HolonomicWithRotation,DiffDrive,KinematicBicycle) at construction. And also have additional custom actions. See https://github.com/proroklab/VectorizedMultiAgentSimulator/pull/76
Control Lyapunov Controllers are available in VMAS as heuristics! @gy2256 made this possible in https://github.com/proroklab/VectorizedMultiAgentSimulator/pull/74
Scenarios that could be done after a reset have been fixed (e.g., transport, dropout) and more kwargs were added to scenarios in https://github.com/proroklab/VectorizedMultiAgentSimulator/pull/78

What's Changed

Added an HeuristicPolicy with Control Lyapunov Controller by @gy2256 in https://github.com/proroklab/VectorizedMultiAgentSimulator/pull/74
[Feature] Actions detached from physics and allow any number of actions by @matteobettini in https://github.com/proroklab/VectorizedMultiAgentSimulator/pull/76
[Feature] Update scenarios by @matteobettini in https://github.com/proroklab/VectorizedMultiAgentSimulator/pull/78

New Contributors

@gy2256 made their first contribution in https://github.com/proroklab/VectorizedMultiAgentSimulator/pull/74

Full Changelog: https://github.com/proroklab/VectorizedMultiAgentSimulator/compare/VMAS-1.3.3...VMAS-1.3.4

- Python
Published by matteobettini over 2 years ago

vmas - VMAS-1.3.3

Fixed bugs related to new vecotrized constraints

Full Changelog: https://github.com/proroklab/VectorizedMultiAgentSimulator/compare/VMAS-1.3.0...VMAS-1.3.3

- Python
Published by matteobettini over 2 years ago

vmas - VMAS-1.3.0

Vectorized physics eginge

If you are using vmas, this is the time to pull.

The physics engine has been rewrote to run vectorised over the agents (not only over environemnts) with no changes at all to the VMAS logic and interface.

The change

Before: vmas was vectorized over the environments, but the engine that resolves collisions and constraints ran a double for loop over the entities
Now: the double for loop over the entities has been replaced by vectorization, making vmas vectorized both in the agent and in the environment dimensions while still keeping the possibility of heterogeneity among the agents.

What it means

Normal environments can observe speedups around 10x
Environments with high density of collidable entities will observe incredible speedups (especially on gpu) up to 10000x
Simulating environments with large number of agents and entities of different shapes should be significantly faster with better computational scaling in the number of entities

Car-like robot dynamics

Thanks to @Jianye-Xu VMAS now has dynamics for car like robots. Meaning that you create your favourite traffic scenarios. Check out the dedicated scenario to see how this works.

What's Changed

[Refactor] use_vmas_env example compatible with 3d actions by @matteobettini in https://github.com/proroklab/VectorizedMultiAgentSimulator/pull/69
[Feature] New dynamics for car-like robots: kinematic bicycle model by @Jianye-Xu in https://github.com/proroklab/VectorizedMultiAgentSimulator/pull/68
[Rendering] Choose cmap to plot by @matteobettini in https://github.com/proroklab/VectorizedMultiAgentSimulator/pull/70
[Perf] Vectorized collisions and other major perf improvements by @matteobettini in https://github.com/proroklab/VectorizedMultiAgentSimulator/pull/71

New Contributors

@Jianye-Xu made their first contribution in https://github.com/proroklab/VectorizedMultiAgentSimulator/pull/68

Full Changelog: https://github.com/proroklab/VectorizedMultiAgentSimulator/compare/VMAS-1.2.13...VMAS-1.3.0

- Python
Published by matteobettini over 2 years ago

vmas - VMAS-1.2.13

Fixes to MPE resetting
Introduces naming convention <name>_<int> for all scenarios (to be used in torchrl for automatic grouping https://github.com/pytorch/rl/pull/1658)

- Python
Published by matteobettini over 2 years ago

vmas - VMAS-1.2.12

What's Changed

[Feature] Text is a Geom and can be returned by scenarios by @matteobettini in https://github.com/proroklab/VectorizedMultiAgentSimulator/pull/54
detach() on input actions
[DOCS] Torchrl notebook link by @matteobettini in https://github.com/proroklab/VectorizedMultiAgentSimulator/pull/59

Full Changelog: https://github.com/proroklab/VectorizedMultiAgentSimulator/compare/VMAS-1.2.11...VMAS-1.2.12

- Python
Published by matteobettini over 2 years ago

vmas - VMAS-1.2.11

Differential drive robot dynamics now available b9199e8b8365b74f2f6ec2d7ffca3ed74ac44941
Observations can be also dictionaries of tensors now 25250a0d011bca5de083710cd44e7a03cf0d7259

- Python
Published by matteobettini about 3 years ago

vmas - VMAS-1.2.10

Many new exciting things in vmas!:

New scenarios: sampling, navigation, wind_flocking
Fixed a rendering memory leak
interactive_rendering.py accepts both scenario names and scenario class
external forces (e.g. gravitiy) can be defined per agent
improved flocking scenario

- Python
Published by matteobettini about 3 years ago

vmas - VMAS-1.2.9

Fix a bug on the viewer device
Now cloning all output and input from vmas simulator
Ready for MAPPO IPPO example in torch rl (https://github.com/pytorch/rl/pull/1027)

- Python
Published by matteobettini about 3 years ago

vmas - VMAS-1.2.8

vmas can now also use dictionaries with agent names as keys for input and output spaces (instead of tuples) https://github.com/proroklab/VectorizedMultiAgentSimulator#input-and-output-spaces

- Python
Published by matteobettini about 3 years ago

vmas - VMAS-1.2.7

Possibility of plotting a function under rendering https://github.com/proroklab/VectorizedMultiAgentSimulator#plot-function-under-rendering
Possibility of passing a scenario class to make_env instead of only being able to pass a scenario name

- Python
Published by matteobettini about 3 years ago