madupite

madupite: A High-Performance Distributed Solver for Large-Scale Markov Decision Processes - Published in JOSS (2025)

Science Score: 100.0%

This score indicates how likely this project is to be science-related based on various indicators:

✓
CITATION.cff file
Found CITATION.cff file
✓
codemeta.json file
Found codemeta.json file
✓
.zenodo.json file
Found .zenodo.json file
✓
DOI references
Found 4 DOI reference(s) in README and JOSS metadata
✓
Academic publication links
Links to: arxiv.org, joss.theoj.org
✓
Committers with academic emails
5 of 8 committers (62.5%) from academic institutions
○
Institutional organization owner
✓
JOSS paper metadata
Published in Journal of Open Source Software

Keywords

cpp dynamic-programming ethz high-performance-computing markov-decision-processes mdp mdpsolver mpi nanobind optimal-control optimization petsc policy-iteration python python3 solver stochastic-optimal-control stochastic-optimization

Last synced: 4 months ago · JSON representation ·

Repository

a High-Performance Distributed Solver for Large-Scale Markov Decision Processes (MDP) relying on Inexact Policy Iteration; for Python and C++

Basic Info

Host: GitHub
Owner: madupite
License: mit
Language: C++
Default Branch: main
Homepage: https://madupite.github.io/
Size: 36.5 MB

Statistics

Stars: 26
Watchers: 1
Forks: 1
Open Issues: 0
Releases: 2

Topics

Created over 1 year ago · Last pushed 9 months ago

Metadata Files

Readme Contributing License Citation

README.md

madupite

a High-Performance Distributed Solver for Large-Scale Markov Decision Processes

madupite is a high-performance C++ library with a Python interface designed for solving large-scale Markov Decision Processes (MDPs) using Distributed Inexact Policy Iteration (iPI)[^1]. Leveraging distributed sparse linear solvers from PETSc, madupite efficiently handles the computational complexity associated with large-scale MDPs.

Key Features

Scalable MDP Solver: Efficiently solve MDPs with large state and action spaces using distributed computation.
Python and C++ APIs: Access the power of madupite through both Python and C++, depending on your performance needs.
Distributed Computing: Integrates with PETSc and MPI for distributed computing on multi-core and cluster environments.

Installation

After cloning the repository, you can install the package using the following commands.

bash conda env create -f environment.yml conda activate madupiteenv pip install .

Note that the conda environment file is limited to Linux for now. After installation, run python examples/install/main.py or mpirun -n <number of ranks> python examples/install/main.py to check if the installation was successful.

We recommend using the provided conda environment. If you prefer to rely on your own environment, make sure to install the required dependencies listed in environment.yml. We refer to the documentation for more information on the installation process.

Examples

Solve any stochastic optimal control problem formulated as a MDP using madupite! For example, the fastest way out of a maze or the optimal control input to balance an inverted pendulum:

Examples on how to use madupite can be found in the documentation (tutorials and examples) as well as in the examples directory.

In short: The transition probabilities and stage costs can either be loaded from files or generated using functions in the code (or a combination of both): * See examples/tutorial/ex1.py for an example on how to generate the required data using function in the code. * See examples/tutorial/ex2.py for an example on how to load the required data from files as well as examples/tutorial/ex2_data_generation.py to see how the required files can be generated from NumPy or SciPy arrays.

The tutorials also provides more detail on the data layout and how to preallocate sparse matrices for improved performance.

Contributing

Contributions to madupite are welcome! Please see the guidelines file CONTRIBUTING.md for more information.

License

madupite is distributed under the MIT License. See the LICENSE file for more information.

Acknowledgements

madupite was developed at the Automatic Control Laboratory at ETH Zurich. This work was supported by the European Research Council under the Horizon 2020 Advanced under Grant 787845 (OCAL) and by the SNSF through NCCR Automation (Grant Number 180545).

If you use madupite in your research, please cite it.

@article{Gargiani2025, doi = {10.21105/joss.07411}, url = {https://doi.org/10.21105/joss.07411}, year = {2025}, publisher = {The Open Journal}, volume = {10}, number = {108}, pages = {7411}, author = {Matilde Gargiani and Philip Pawlowsky and Robin Sieber and Václav Hapla and John Lygeros}, title = {madupite: A High-Performance Distributed Solver for Large-Scale Markov Decision Processes}, journal = {Journal of Open Source Software} }

[^1]: Gargiani, M.; Sieber. R.; Balta, E.; Liao-McPherson, D.; Lygeros, J. Inexact Policy Iteration Methods for Large-Scale Markov Decision Processes. https://arxiv.org/abs/2404.06136.

Owner

Name: madupite
Login: madupite
Kind: organization

Repositories: 1
Profile: https://github.com/madupite

JOSS Publication

madupite: A High-Performance Distributed Solver for Large-Scale Markov Decision Processes

Published

April 02, 2025

DOI

10.21105/joss.07411

Volume 10, Issue 108, Page 7411

Authors

Matilde Gargiani

Automatic Control Laboratory (IfA), ETH Zurich, 8092 Zurich, Switzerland

Philip Pawlowsky

Automatic Control Laboratory (IfA), ETH Zurich, 8092 Zurich, Switzerland

Robin Sieber

Automatic Control Laboratory (IfA), ETH Zurich, 8092 Zurich, Switzerland

Václav Hapla

Department of Earth and Planetary Sciences, ETH Zurich, 8092 Zurich, Switzerland, Department of Applied Mathematics, FEECS at VSB-TU Ostrava, Czechia

John Lygeros

Automatic Control Laboratory (IfA), ETH Zurich, 8092 Zurich, Switzerland

Editor

Tristan Miller

Citation (CITATION.cff)

cff-version: "1.2.0"
authors:
- family-names: Gargiani
  given-names: Matilde
  orcid: "https://orcid.org/0000-0001-8615-6214"
- family-names: Pawlowsky
  given-names: Philip
  orcid: "https://orcid.org/0009-0003-9732-3884"
- family-names: Sieber
  given-names: Robin
  orcid: "https://orcid.org/0009-0002-8592-8387"
- family-names: Hapla
  given-names: Václav
  orcid: "https://orcid.org/0000-0002-9190-2207"
- family-names: Lygeros
  given-names: John
  orcid: "https://orcid.org/0000-0002-6159-1962"
contact:
- family-names: Gargiani
  given-names: Matilde
  orcid: "https://orcid.org/0000-0001-8615-6214"
- family-names: Pawlowsky
  given-names: Philip
  orcid: "https://orcid.org/0009-0003-9732-3884"
- family-names: Sieber
  given-names: Robin
  orcid: "https://orcid.org/0009-0002-8592-8387"
- family-names: Lygeros
  given-names: John
  orcid: "https://orcid.org/0000-0002-6159-1962"
doi: 10.5281/zenodo.15105482
message: If you use this software, please cite our article in the
  Journal of Open Source Software.
preferred-citation:
  authors:
  - family-names: Gargiani
    given-names: Matilde
    orcid: "https://orcid.org/0000-0001-8615-6214"
  - family-names: Pawlowsky
    given-names: Philip
    orcid: "https://orcid.org/0009-0003-9732-3884"
  - family-names: Sieber
    given-names: Robin
    orcid: "https://orcid.org/0009-0002-8592-8387"
  - family-names: Hapla
    given-names: Václav
    orcid: "https://orcid.org/0000-0002-9190-2207"
  - family-names: Lygeros
    given-names: John
    orcid: "https://orcid.org/0000-0002-6159-1962"
  date-published: 2025-04-02
  doi: 10.21105/joss.07411
  issn: 2475-9066
  issue: 108
  journal: Journal of Open Source Software
  publisher:
    name: Open Journals
  start: 7411
  title: "madupite: A High-Performance Distributed Solver for
    Large-Scale Markov Decision Processes"
  type: article
  url: "https://joss.theoj.org/papers/10.21105/joss.07411"
  volume: 10
title: "madupite: A High-Performance Distributed Solver for Large-Scale
  Markov Decision Processes"

GitHub Events

Total

Release event: 1
Watch event: 6
Issue comment event: 1
Push event: 12
Pull request event: 3
Pull request review comment event: 1
Pull request review event: 2
Fork event: 2
Create event: 2

Last Year

Release event: 1
Watch event: 6
Issue comment event: 1
Push event: 12
Pull request event: 3
Pull request review comment event: 1
Pull request review event: 2
Fork event: 2
Create event: 2

Committers

Last synced: 5 months ago

All Time

Total Commits: 503
Total Committers: 8
Avg Commits per committer: 62.875
Development Distribution Score (DDS): 0.431

Past Year

Commits: 166
Committers: 7
Avg Commits per committer: 23.714
Development Distribution Score (DDS): 0.633

Top Committers

Name	Email	Commits
Robin Sieber	r**2@h**m	286
Philip Pawlowsky	p**y@g**m	90
Vaclav Hapla	v**a@e**h	70
gmatilde	g**e@i**h	42
gmatilde	g**e@e**h	6
Matilde Gargiani	m**i@g**m	4
Philip Pawlowsky	p**y@e**h	4
Daniel S. Katz	d**z@i**g	1

Committer Domains (Top 20 + Academic)

ethz.ch: 2 ieee.org: 1 itet-ifa-601.ethz.ch: 1 erdw.ethz.ch: 1

Issues and Pull Requests

Last synced: 4 months ago

All Time

Total issues: 0
Total pull requests: 8
Average time to close issues: N/A
Average time to close pull requests: 3 days
Total issue authors: 0
Total pull request authors: 3
Average comments per issue: 0
Average comments per pull request: 0.13
Merged pull requests: 8
Bot issues: 0
Bot pull requests: 0

Past Year

Issues: 0
Pull requests: 8
Average time to close issues: N/A
Average time to close pull requests: 3 days
Issue authors: 0
Pull request authors: 3
Average comments per issue: 0
Average comments per pull request: 0.13
Merged pull requests: 8
Bot issues: 0
Bot pull requests: 0

View more stats

Top Authors

Issue Authors

Pull Request Authors

rjs02 (3)
philippawlowsky (3)
danielskatz (2)

Top Labels

Issue Labels

Pull Request Labels

Dependencies

.github/workflows/ci-pipeline.yml actions

JamesIves/github-pages-deploy-action v4 composite
actions/checkout v2 composite
conda-incubator/setup-miniconda v2 composite

pyproject.toml pypi

environment.yml conda

cmake
gcc_linux-64 13.2.0.*
gxx_linux-64 13.2.0.*
make
matplotlib
mpich 4.2.1.*
nanobind
numba
numpy
petsc >=3.21.0,<3.22.0
pip
pkgconfig
pre-commit
pydata-sphinx-theme
pytest
python
scipy
sphinx
sphinxcontrib-video
sysroot_linux-64 2.28.*

madupite

Science Score: 100.0%

Keywords

Repository

Basic Info

Statistics

Topics

Metadata Files

README.md

madupite

Key Features

Installation

Examples

Contributing

License

Acknowledgements

Owner

JOSS Publication

madupite: A High-Performance Distributed Solver for Large-Scale Markov Decision Processes

Authors

Editor

Tags

Citation (CITATION.cff)

GitHub Events

Total

Last Year

Committers

All Time

Past Year

Top Committers

Committer Domains (Top 20 + Academic)

Issues and Pull Requests

All Time

Past Year

Top Authors

Issue Authors

Pull Request Authors

Top Labels

Issue Labels

Pull Request Labels

Dependencies