Devito: Fast Stencil Computation from Symbolic Specification

DEVITO SENAI CIMATEC is a DSL developed based on the open-source Devito package Devito: a Python package to implement optimized stencil computation (e.g., finite differences, image processing, machine learning) from high-level symbolic problem definitions. This DSL builds on SymPy and employs automated code generation and just-in-time compilation to execute optimized computational kernels on several computer platforms, including CPUs, GPUs, and clusters thereof.

• About Devito
• Disk Swap
• Installation
• Resources
• FAQs
• Performance

About Devito

Devito provides a functional language to implement sophisticated operators that can be made up of multiple stencil computations, boundary conditions, sparse operations (e.g., interpolation), and much more. A typical use case is explicit finite difference methods for approximating partial differential equations. For example, a 2D diffusion operator may be implemented with Devito as follows

```python

grid = Grid(shape=(10, 10)) f = TimeFunction(name='f', grid=grid, space_order=2) eqn = Eq(f.dt, 0.5 * f.laplace) op = Operator(Eq(f.forward, solve(eqn, f.forward))) ```

An Operator generates low-level code from an ordered collection of Eq (the example above being for a single equation). This code may also be compiled and executed

```python

op(t=timesteps, dt=dt) ```

There is virtually no limit to the complexity of an Operator -- the Devito compiler will automatically analyze the input, detect and apply optimizations (including single- and multi-node parallelism), and eventually generate code with suitable loops and expressions.

Key features include:

• A functional language to express finite difference operators.
• Straightforward mechanisms to adjust the discretization.
• Constructs to express sparse operators (e.g., interpolation), classic linear operators (e.g., convolutions), and tensor contractions.
• Seamless support for boundary conditions and adjoint operators.
• A flexible API to define custom stencils, sub-domains, sub-sampling, and staggered grids.
• Generation of highly optimized parallel code (SIMD vectorization, CPU and GPU parallelism via OpenMP and OpenACC, multi-node parallelism via MPI, blocking, aggressive symbolic transformations for FLOP reduction, etc.).
• Distributed NumPy arrays over multi-node (MPI) domain decompositions.
• Inspection and customization of the generated code.
• Autotuning framework to ease performance tuning.
• Smooth integration with popular Python packages such as NumPy, SymPy, Dask, and SciPy, as well as machine learning frameworks such as TensorFlow and PyTorch.

Disk Swap

The Disk Swap is an exclusive feature of DEVITO SENAI CIMATEC that enables the full storage of TimeFunctions (commonly used to represent wavefields in various models) on high-performance devices, such as NVMe drives.

This functionality overcomes the field size limitations imposed by executions relying solely on RAM storage, as well as the poor performance of alternative approaches employing checkpointing techniques.

DEVITO SENAI CIMATEC provides an extremely simple and functional configuration interface for Disk Swap, allowing its setup and application in just a few lines of code.

```python

grid = Grid(shape=(10, 10)) f = TimeFunction(name='f', grid=grid, spaceorder=2) eqn = Eq(f.dt, 0.5 * f.laplace) dsconfig = DiskSwapConfig(functions=[f], mode="write", path="pathtodevice") op = Operator(Eq(f.forward, solve(eqn, f.forward)), opt=('advanced', {'disk-swap': ds_config}) ```

The configuration, for example, for an operator that stores the wavefield to disk, can be done intuitively and without increasing the code complexity for the user, as demonstrated above.

Installation

The use of virtual environments is recommended to isolate package dependencies, ensuring that the installation of DEVITO does not interfere with other projects or system configurations.

In addition to the installation of mandatory packages required for the basic functionality of the tool, DEVITO also supports the installation of additional dependencies for extended features, organized into four groups:

• extras: dependencies for Jupyter notebooks, plotting, and benchmarking.
• tests: dependencies for the testing infrastructure.
• mpi: dependencies for the MPI infrastructure.
• nvidia: dependencies to enable GPU execution.

venv install: ```

Creation

python -m venv

Activation

source /bin/activate # Para sistemas Linux/macOS

Installing additional dependencies

pip install git+https://codigo-externo.petrobras.com.br/senai-cimatec-lde/devito.git

...or to install it with the additional dependencies already included:

pip install devito[tests,extras,nvidia,mpi] @ git+https://codigo-externo.petrobras.com.br/senai-cimatec-lde/devito.git@main ```

conda install: ```

Creation

conda create --name

Activation

conda activate

Download repository

git clone https://codigo-externo.petrobras.com.br/senai-cimatec-lde/devito.git

Install

cd devito pip install -e .

...or to install it with the additional dependencies already included:

pip install -e .[extras,mpi,nvidia,tests] ```

Resources

To learn how to use the DEVITO SENAI CIMATEC, here is a good place to start, with lots of examples and tutorials.

The original Devito website also provides access to other information, including documentation and also a FAQs are discussed here.

Performance

If you are interested in any of the following

• Generation of parallel code (CPU, GPU, multi-node via MPI);
• Performance tuning;
• Benchmarking operators;

then you should take a look at this README.