Ratel: Performance portable solid mechanics with libCEED and PETSc

Ratel: Performance portable solid mechanics with libCEED and PETSc - Published in JOSS (2026)

https://gitlab.com/micromorph/ratel

Keywords

Keywords from Contributors

Repository

Extensible, performance-portable solid mechanics with libCEED and PETSc

https://gitlab.com/micromorph/ratel/blob/main/

# Ratel: Extensible, performance-portable solid mechanics

[![GitLab-CI](https://gitlab.com/micromorph/ratel/badges/main/pipeline.svg?key_text=GitLab-CI)](https://gitlab.com/micromorph/ratel/-/pipelines?page=1&scope=all&ref=main)
[![BSD-2-Clause](https://img.shields.io/badge/License-BSD%202--Clause-orange.svg)](https://opensource.org/licenses/BSD-2-Clause)
[![Documentation](https://img.shields.io/badge/Documentation-latest-blue.svg)](https://ratel.micromorph.org)
[![Code Coverage](https://gitlab.com/micromorph/ratel/badges/main/coverage.svg)](https://gitlab.com/micromorph/ratel/-/commits/main)
[![JOSS Paper](https://joss.theoj.org/papers/31e088c09cfc1b524ddd7a446d529789/status.svg)](https://joss.theoj.org/papers/31e088c09cfc1b524ddd7a446d529789)


Ratel is a solid mechanics library and applications based on [libCEED](https://libceed.org) and [PETSc](https://petsc.org) with support for efficient high-order elements and CUDA and ROCm GPUs.

Solid mechanics simulations provide vital information for many engineering applications, using a large amount of computational resources from workstation to supercomputing scales.
The industry standard for implicit analysis uses assembled sparse matrices with low-order elements, typically $Q_1$ hexahedral and $P_2$ tetrahedral elements, with the linear systems solved using sparse direct solvers, algebraic multigrid, or multilevel domain decomposition.
This approach has two fundamental inefficiencies: poor approximation accuracy per Degree of Freedom (DoF) and high computational and memory cost per DoF due to choice of data structures and algorithms.
High-order finite elements implemented in a matrix-free fashion with appropriate preconditioning strategies can overcome these inefficiencies.

For further details on the benefits of high-order, matrix-free finite elements for solid mechanics, see [our preprint on arXiv](https://arxiv.org/abs/2204.01722).

# Getting Started


Ratel provides static, quasistatic, and dynamic applications with a variety of elastic material models.
Additionally, Ratel has an API for creation of solid mechanics applications with PETSc.
The full library documentation may be found at [https://ratel.micromorph.org/](https://ratel.micromorph.org/).

## Docker

Docker images provide a 'quick start' option for users wanting to use the Ratel quasistatic and dynamic applications.
These Docker images are automatically generated for the latest commit to `main`.

To run the Ratel quasistatic application in the published Docker container with a local work directory, use:

``` console
host$ docker run -it --rm -v $(pwd):/work registry.gitlab.com/micromorph/ratel
container$ ratel-quasistatic -options_file config.yaml
```

The published Ratel Docker image provides a generic CPU only build of Ratel.
For GPU support or builds optimized to your specific hardware, use the download and build instructions below.

## Download and Install

A local build and installation provides greater control over build options and optimization.
Ratel is open-source and can be downloaded from [the Ratel repository on GitLab](https://gitlab.com/micromorph/ratel/).

```console
$ git clone https://gitlab.com/micromorph/ratel
```

### Prerequisites

The Ratel solid mechanics library is based upon libCEED and PETSc.

#### libCEED

Ratel requires libCEED v1.0.0 or libCEED's `main` development branch, which can be [cloned from Github](https://github.com/CEED/libCEED).

```console
$ git clone https://github.com/CEED/libCEED
$ make -j8 -C libCEED
```

The above will be enough for most simple CPU installations; see the [libCEED documentation](https://libceed.org/en/latest/gettingstarted/#) for details on using GPUs, tuning, and more complicated environments.

The current minimum required libCEED commit hash can be found in `containers/docker/libceed/Dockerfile`.

#### PETSc

Ratel requires PETSC v3.24.3 or PETSc's `main` development branch, which can be [cloned from GitLab](https://gitlab.com/petsc/petsc).

```console
$ git clone https://gitlab.com/petsc/petsc
```

Follow the [PETSc documentation](https://petsc.org/main/install/) to configure and build PETSc.
It is recommended to install with the CGNS [external package](https://petsc.org/release/install/install/#external-packages) with the option `--download-cgns`, as CGNS offers high order geometry in output files.

The current minimum required PETSc commit hash can be found in `containers/docker/petsc/Dockerfile`.

#### Automatic Differentiation

Ratel supports developing new constitutive models with Automatic Differentiation tools, including Enzyme-AD and ADOL-C.

##### Enzyme-AD

Enzyme-AD can be [cloned from GitHub](https://github.com/EnzymeAD/Enzyme).
Enzyme-AD is based upon the LLVM toolchain and requires the Clang compiler.

```console
$ git clone https://github.com/EnzymeAD/Enzyme
```

Follow the [Enzyme documentation](https://enzyme.mit.edu/Installation/) to build Enzyme.

##### ADOL-C

ADOL-C can be [cloned from GitHub](https://github.com/coin-or/ADOL-C).

```console
$ git clone https://github.com/coin-or/ADOL-C
```

Follow the [ADOL-C instructions](https://github.com/coin-or/ADOL-C/blob/master/README.md) to build ADOL-C.

### Building

The environment variables `CEED_DIR`, `PETSC_DIR`, and `PETSC_ARCH` must be set to build Ratel.
Assuming you have cloned the Ratel repository as above, export these environment variables:

```console
$ export CEED_DIR=[path to libCEED] PETSC_DIR=[path to PETSc] PETSC_ARCH=[PETSc arch]
```

If you are using Enzyme-AD, set the `ENZYME_LIB` environment variable as well:

```console
$ export ENZYME_LIB=[path to Enzyme]
```

For ADOL-C, you can either set `ADOLC_LIB` and `ADOLC_INCLUDE` directly, or set `ADOLC_DIR` (which should contain both `include/` and `lib/` subdirectories).
Alternatively, if ADOL-C is installed globally, you can simply build Ratel with `WITH_ADOLC=1`.

```console
$ export ADOLC_LIB=[path to ADOL-C lib directory] ADOLC_LIB=[path to ADOL-C include directory]
```
or

```console
$ export ADOLC_DIR=[path to ADOL-C directory]
```

**Note:** Building Ratel with Enzyme-AD or ADOL-C is optional.
Depending on your system setup, you might not be able to build both at the same time.

Build Ratel with:
```console
$ make -j8
```

To run a sample problem with the [quasistatic example](example-quasistatic) using multiple pseudotimesteps, run:

```console
$ bin/ratel-quasistatic -options_file examples/ymls/ex02/schwarz-pendulum.yml -view_output_fields cgns:output.cgns
```

To activate common solver monitor options such as solution values at each pseudotimestep (that can be visualized using software like Paraview or VisIt), run:


```console
$ bin/ratel-quasistatic -options_file examples/ymls/ex02/schwarz-pendulum.yml -options_file examples/ymls/ratel-monitor.yml
```

To test the installation, use

```console
$ make test -j8
```

See the {ref}`examples` for instructions on using the Ratel applications.

### Install

To install Ratel, run:

```console
$ make install prefix=/path/to/install/dir
```

or (e.g., if creating packages):

```console
$ make install prefix=/usr DESTDIR=/packaging/path
```

To build and install in separate steps, run:

```console
$ make for_install=1 prefix=/path/to/install/dir
$ make install prefix=/path/to/install/dir
```

The usual variables like `CEED_DIR` and `PETSC_DIR` are used.
Use `STATIC=1` to build static libraries (`ratel.a`).

#### pkg-config

In addition to library and header, Ratel provides a [pkg-config](https://en.wikipedia.org/wiki/Pkg-config) file that can be used to easily compile and link.
[For example](https://people.freedesktop.org/~dbn/pkg-config-guide.html#faq), if `$prefix` is a standard location or you set the environment variable `PKG_CONFIG_PATH`:

```console
$ cc `pkg-config --cflags --libs ratel` -o myapp myapp.c
```

will build `myapp` with Ratel.
This can be used with the source or installed directories.
Most build systems have support for pkg-config.

### GPU Support

Ratel supports CUDA and ROCm GPUs.
To use these features, build PETSc and libCEED with GPU support and run with `-ceed /gpu/cuda` or `-ceed /gpu/hip`.

The multigrid coarse solver uses PETSc [PCGAMG](https://petsc.org/main/docs/manualpages/PC/PCGAMG.html) by default.
Other coarse solvers such as [PCHYPRE](https://petsc.org/main/docs/manualpages/PC/PCHYPRE.html) can be selected with `-mg_coarse_pc_type hypre`.
Coarse solvers generally require an assembled matrix, and this be done on the GPU with suitable matrix types.

The default multigrid coarse solver options are set based upon the libCEED backend selected at runtime.
These options are listed below:

#### Default configurations

| **libCEED Backend** | **Default DM options**                              | **Notes**                                            | **PETSc configure**                                      |
| ------------------- | --------------------------------------------------- | ---------------------------------------------------- | -------------------------------------------------------- |
| `-ceed /cpu/self`   | `-coarse_dm_mat_type aij`                           | assembles coarse matrix in CPU memory; GAMG uses CPU | None                                                     |
| `-ceed /gpu/cuda`   | `-coarse_dm_mat_type aijcusparse -dm_vec_type cuda` | GAMG and Hypre use GPU                               | `--with-cuda`                                            |
| `-ceed /gpu/hip`    | `-coarse_dm_mat_type aijkokkos -dm_vec_type kokkos` | GAMG and Hypre use GPU                               | `--with-hip --download-kokkos --download-kokkos-kernels` |

## Spack

Ratel can also be installed via Spack.
Follow the [Spack documentation](https://spack-tutorial.readthedocs.io) to install packages.

For GPU support, the Ratel Spack package supports the options `+cuda` and `+rocm`.
For example, `spack install ratel@develop+cuda cuda_arch=XX`.


## How to Cite

The archival copy of the Ratel user manual is maintained on [Zenodo](https://doi.org/10.5281/zenodo.18508986).
To cite the user manual:

```bibtex
@misc{ratel-user-manual,
  author    = {Atkins, Zachary R and
               Brown, Jed and
               Di Gioacchino, Fabio and
               Ghaffari, Layla and
               Irwin, Zachariah T and
               Shakeri, Rezgar and
               Stengel, Ren and
               Thompson, Jeremy L},
  title     = {Ratel User Manual},
  month     = feb,
  year      = 2026,
  publisher = {Zenodo},
  version   = {v1.0.01},
  doi       = {10.5281/zenodo.15213258},
  url       = {https://doi.org/10.5281/zenodo.15213258}
}
```

## Contact

You can reach the Ratel team by leaving a comment in the [issue tracker](https://gitlab.com/micromorph/ratel/-/issues).

## Copyright

The following copyright applies to each file in the Ratel software suite, unless otherwise stated in the file:

> Copyright (c) 2021-2025 University of Colorado Boulder, Lawrence Livermore National Security LLC, and other authors.
> All rights reserved.

See files LICENSE and NOTICE for details.

Owner

JOSS Publication

Ratel: Performance portable solid mechanics with libCEED and PETSc

Published

February 12, 2026

DOI

10.21105/joss.08388

Volume 11, Issue 118, Page 8388

Authors

Zachary R. Atkins
University of Colorado at Boulder

Jed Brown
University of Colorado at Boulder

Fabio Di Gioacchino
University of Colorado at Boulder

Layla Ghaffari
University of Colorado at Boulder

Zachariah T. Irwin
University of Colorado at Boulder

Rezgar Shakeri
University of Colorado at Boulder

Karen Stengel
University of Colorado at Boulder

Jeremy L. Thompson
University of Colorado at Boulder

Editor

Mojtaba Barzegari

Tags

high-performance computing high-order methods finite elements matrix-free solid mechanics

View PDF Review Thread Software Archive

Committers

Last synced: about 2 years ago

Issues and Pull Requests

Last synced: 8 months ago