charm

The Charm++ parallel programming system. Visit https://charmplusplus.org/ for more information.

https://github.com/charmplusplus/charm

Keywords

Repository

The Charm++ parallel programming system. Visit https://charmplusplus.org/ for more information.

README.ampi

Adaptive MPI (AMPI)
-------------------
AMPI is an implementation of the MPI standard written on top of Charm++, meant
to give MPI applications access to high-level, application-independent features
such as overdecomposition (processor virtualization), dynamic load balancing,
automatic fault tolerance, and overlap of computation and communication. For
more information on all topics related to AMPI, consult the AMPI manual here:

    http://charm.cs.illinois.edu/manuals/html/ampi/manual.html


Building AMPI
-------------
AMPI has its own target in the build system. You can run the top-level
build script interactively using "./build", or you can specify your
architecture, operating system, compilers, and other options directly.
For example:

    ./build AMPI netlrts-linux-x86_64 gfortran gcc --with-production


Compiling and Linking AMPI Programs
-----------------------------------
AMPI source files can be compiled and linked with the wrappers found
in bin/, such as ampicc, ampicxx, ampif77, and ampif90:

    ampif90 pgm.f90 -o pgm

For consistency with other MPI implementations, these wrappers are also
provided using their standard names with the suffix ".ampi".
Additionally, the "bin/ampi" subdirectory contains the wrappers with
their standard names, for simplicity of overriding the default system MPI
via the $PATH environment variable.


Running AMPI Programs
---------------------
AMPI programs can be run with charmrun like any other Charm++ program. In
addition to the number of processes, specified with "+p n", AMPI programs
also take the total number of virtual processors (VPs) to run with as "+vp n".
For example, to run an AMPI program 'pgm' on 4 processors using 32 ranks, do:

    ./charmrun +p 4 ./pgm +vp 32

To run with dynamic load balancing, add "+balancer ":

    ./charmrun +p 4 ./pgm +vp 32 +balancer RefineLB


Porting to AMPI
---------------
Global and static variables are unsafe for use in virtualized AMPI programs.
This is because globals are defined at the process level, and AMPI ranks are
implemented as user-level threads, which may share a process with other ranks
Therefore, to run with more than 1 VP per processor, all globals and statics
that are non-readonly and whose value does not depend on rank must be modified
to use local storage. Consult the AMPI manual for more information on global
variable privatization and automated approaches to privatization.

AMPI programs must have the following main function signatures, so that AMPI
can bootstrap before invoking the user's main function:
    * C/C++ programs should use "int main(int argc, char **argv)"
    * Fortran programs must use "Subroutine MPI_Main" instead of
      "Program Main"


Incompatibilities and Extensions
--------------------------------
AMPI has some known flaws and incompatibilities with other MPI implementations:
    * RMA routines do not have support for derived datatypes.
    * Not all collectives are supported on intercommunicators.
    * No support for MPI_Pack_external, MPI_Pack_external_size, MPI_Unpack_external.

AMPI also has extensions to the MPI standard to enable use of the high-level
features provided by the Charm++ adaptive runtime system. All extensions are
prefixed with AMPI_:
    * AMPI_Migrate tells the runtime system that the application has reached a
      point at which the runtime system may serialize and migrate ranks.
      It is used for dynamic load balancing and fault tolerance. See the AMPI
      manual for more information on how to use it.
    * AMPI_Register_pup is used to register PUP routines and user data.
    * AMPI_Get_pup_data returns a pointer to user data managed by the runtime.
    * AMPI_Load_set_value sets the calling rank's load to the given user value.
    * AMPI_Load_start_measure starts load balance information collection.
    * AMPI_Load_stop_measure stops load balance information collection.
    * AMPI_Load_reset_measure clears the load balance database.
    * AMPI_Migrate_to_pe migrates the calling rank to the given PE.
    * AMPI_Set_migratable sets the migratability of the calling rank.
    * AMPI_Command_argument_count returns the number of command line arguments
      given to a Fortran AMPI program excluding charmrun and AMPI parameters.
    * AMPI_Get_command_argument returns an argument from the command line
      to a Fortran AMPI program.

MPI-IO is support is available via our port of the ROMIO library. however:
    * ROMIO is not built by default due to the fact that the current port is
      incompatible with GCC 14 and beyond.  Add --with-romio to your
      build line to enable MPI-IO support via ROMIO.

Note that AMPI defines a preprocessor symbol "AMPI" so that user codes can
check for AMPI's presence at compile time using "#ifdef AMPI".

Owner

Citation (CITATION.cff)

# YAML 1.2
# Note that we have omitted the release date and version due to using the umbrella DOI
---
cff-version: "1.0.3"
title: "The Charm++ Parallel Programming System"
authors:
  - family-names: "Kale"
    given-names: "Laxmikant V."
    affiliation: "University of Illinois at Urbana-Champaign"
message: "If you use this software, please cite both the software and the given book chapter."
repository-code: "https://github.com/charmplusplus/charm"
doi: "10.5281/zenodo.3370873"
notes: |
  @misc{charm-zenodo,
    author = {Kale, Laxmikant and Acun, Bilge and Bak, Seonmyeong and Becker, Aaron and
      Bhandarkar, Milind and Bhat, Nitin and Bhatele, Abhinav and Bohm, Eric and
      Bordage, Cyril and Brunner, Robert and Buch, Ronak and Chakravorty, Sayantan
      and Chandrasekar, Kavitha and Choi, Jaemin and Denardo, Michael and DeSouza,
      Jayant and Diener, Matthias and Dokania, Harshit and Dooley, Isaac and Fenton,
      Wayne and Galvez, Juan and Gioachin, Fillipo and Gupta, Abhishek and Gupta,
      Gagan and Gupta, Manish and Gursoy, Attila and Harsh, Vipul and Hu, Fang and
      Huang, Chao and Jagathesan, Narain and Jain, Nikhil and Jetley, Pritish and
      Jindal, Prateek and Kanakagiri, Raghavendra and Koenig, Greg and Krishnan,
      Sanjeev and Kumar, Sameer and Kunzman, David and Lang, Michael and Langer,
      Akhil and Lawlor, Orion and Wai Lee, Chee and Lifflander, Jonathan and Mahesh,
      Karthik and Mendes, Celso and Menon, Harshitha and Mei, Chao and Meneses,
      Esteban and Mikida, Eric and Miller, Phil and Mokos, Ryan and Narayanan,
      Venkatasubrahmanian and Ni, Xiang and Nomura, Kevin and Paranjpye, Sameer and
      Ramachandran, Parthasarathy and Ramkumar, Balkrishna and Ramos, Evan and
      Robson, Michael and Saboo, Neelam and Saletore, Vikram and Sarood, Osman and
      Senthil, Karthik and Shah, Nimish and Shu, Wennie and B. Sinha, Amitabh and
      Sun, Yanhua and Sura, Zehra and Totoni, Ehsan and Varadarajan, Krishnan and
      Venkataraman, Ramprasad and Wang, Jackie and Wesolowski, Lukasz and White, Sam
      and Wilmarth, Terry and Wright, Jeff and Yelon, Joshua and Zheng, Gengbin},
    doi = {10.5281/zenodo.3370873},
    month = {Aug},
    title = {{The Charm++ Parallel Programming System}},
    url = {https://charm.cs.illinois.edu},
    year = {2019},
  }
references:
  - type: "book"
    scope: "Cite this book chapter to refer to the concepts underlying Charm++."
    authors:
      - family-names: "Kale"
        given-names: "Laxmikant V."
        affiliation: "Department of Computer Science, University of Illinois at Urbana-Champaign"
      - family-names: "Zheng"
        given-names: "Gengbin"
        affiliation: "National Center for Supercomputing Applications, University of Illinois at Urbana-Champaign"
    doi: "10.1201/b16251"
    editors:
      - family-names: "Kale"
        given-names: "Laxmikant V."
      - family-names: "Bhatele"
        given-names: "Abhinav"
    publisher:
      name: "CRC Press"
      city: "Boca Raton"
    section: "Chapter 1: The Charm++ Programming Model"
    title: "Parallel Science and Engineering Applications: The Charm++ Approach"
    url: "https://charm.cs.illinois.edu/newPapers/13-46/paper.pdf"
    year: 2013
    notes: |
      @incollection{Kale2013,
        address = {Boca Raton, FL, USA},
        author = {Kale, Laxmikant V. and Zheng, Gengbin},
        booktitle = {{Parallel Science and Engineering Applications: The Charm++ Approach}},
        chapter = 1,
        doi = {10.1201/b16251},
        edition = {1st},
        editor = {Kale, Laxmikant V. and Bhatele, Abhinav},
        isbn = {1466504129, 9781466504127},
        pages = {1-16},
        publisher = {CRC Press, Inc.},
        title = {{Chapter 1: The Charm++ Programming Model}},
        year = {2013},
      }

GitHub Events

Issues and Pull Requests

Last synced: 6 months ago