Home    Install    Documentation    Cray Labs    Contact    Join us on Slack!   

SmartSim

SmartSim is made up of two parts 1. SmartSim Infrastructure Library (This repository) 2. SmartRedis

The two library components are designed to work together, but can also be used independently.

SmartSim is a workflow library that makes it easier to use common Machine Learning (ML) libraries, like PyTorch and TensorFlow, in High Performance Computing (HPC) simulations and applications. SmartSim launches ML infrastructure on HPC systems alongside user workloads.

SmartRedis provides an API to connect HPC workloads, particularly (MPI + X) simulations, to the ML infrastructure, namely the The Orchestrator database, launched by SmartSim.

Applications integrated with the SmartRedis clients, written in Fortran, C, C++ and Python, can send data to and remotely request SmartSim infrastructure to execute ML models and scripts on GPU or CPU. The distributed Client-Server paradigm allows for data to be seamlessly exchanged between applications at runtime without the utilization of MPI.

Table of Contents - SmartSim - Quick Start - SmartSim Infrastructure Library - Experiments - Hello World - Hello World MPI - Experiments on HPC Systems - Interactive Launch Example - Batch Launch Examples - Infrastructure Library Applications - Redis + RedisAI - Local Launch - Interactive Launch - Batch Launch - SmartRedis - Tensors - Datasets - SmartSim + SmartRedis Tutorials - Run the Tutorials - Online Analysis - Lattice Boltzmann Simulation - Online Processing - Singular Value Decomposition - Online Inference - PyTorch CNN Example - Publications - Cite - bibtex

Quick Start

The documentation has a number of tutorials that make it easy to get used to SmartSim locally before using it on your system. Each tutorial is a Jupyter notebook that can be run through the SmartSim Tutorials docker image which will run a jupyter lab with the tutorials, SmartSim, and SmartRedis installed.

```bash docker pull ghcr.io/craylabs/smartsim-tutorials:latest docker run -p 8888:8888 ghcr.io/craylabs/smartsim-tutorials:latest

click on link to open jupyter lab

```

SmartSim Infrastructure Library

The Infrastructure Library (IL), the smartsim python package, facilitates the launch of Machine Learning and simulation workflows. The Python interface of the IL creates, configures, launches and monitors applications.

Experiments

The Experiment object is the main interface of SmartSim. Through the Experiment users can create references to user applications called Models.

Hello World

Below is a simple example of a workflow that uses the IL to launch hello world program using the local launcher which is designed for laptops and single nodes.

```python from smartsim import Experiment

exp = Experiment("simple", launcher="local")

settings = exp.createrunsettings("echo", exeargs="Hello World") model = exp.createmodel("hello_world", settings)

exp.start(model, block=True) print(exp.get_status(model)) ```

Hello World MPI

The Experiment.createrunsettings method returns a RunSettings object which defines how a model is launched. There are many types of RunSettings supported by SmartSim.

• RunSettings
• MpirunSettings
• SrunSettings
• AprunSettings

The following example launches a hello world MPI program using the local launcher for single compute node, workstations and laptops.

```Python from smartsim import Experiment

exp = Experiment("helloworld", launcher="local") mpisettings = exp.createrunsettings(exe="echo", exeargs="Hello World!", runcommand="mpirun") mpisettings.settasks(4)

mpimodel = exp.createmodel("helloworld", mpisettings)

exp.start(mpimodel, block=True) print(exp.getstatus(model)) ```

If an argument of run_command="auto" (the default) is passed to Experiment.create_run_settings, SmartSim will attempt to find a run command on the system with which it has a corresponding RunSettings class. If one can be found, Experiment.create_run_settings will instance and return an object of that type.

Experiments on HPC Systems

SmartSim integrates with common HPC schedulers providing batch and interactive launch capabilities for all applications:

• Slurm
• SGE
• PBSPro
• Local (for laptops/single node, no batch)

In addition, on Slurm and PBS systems, Dragon can be used as a launcher. Please refer to the documentation for instructions on how to insall it on your system and use it in SmartSim.

Interactive Launch Example

The following launches the same hello_world model in an interactive allocation.

```bash

get interactive allocation (Slurm)

salloc -N 3 --ntasks-per-node=20 --ntasks 60 --exclusive -t 00:10:00

get interactive allocation (PBS)

qsub -l select=3:ncpus=20 -l walltime=00:10:00 -l place=scatter -I -q

```

This same script will run on a SLURM, PBS, or SGE system as the launcher is set to auto in the Experiment initialization. The run command like mpirun, aprun or srun will be automatically detected from what is available on the system.

```python

hello_world.py

from smartsim import Experiment

exp = Experiment("helloworldexp", launcher="auto") run = exp.createrunsettings(exe="echo", exeargs="Hello World!") run.settasks(60) run.settasksper_node(20)

model = exp.createmodel("helloworld", run) exp.start(model, block=True, summary=True)

print(exp.get_status(model)) bash

in interactive terminal

python hello_world.py ```

This script could also be launched in a batch file instead of an interactive terminal. For example, for Slurm:

```bash

!/bin/bash

SBATCH --exclusive

SBATCH --nodes=3

SBATCH --ntasks-per-node=20

SBATCH --time=00:10:00

python /path/to/hello_world.py bash

on Slurm system

sbatch runhelloworld.sh ```

Batch Launch Examples

SmartSim can also launch workloads in a batch directly from Python, without the need for a batch script. Users can launch groups of Model instances in a Ensemble.

The following launches 4 replicas of the the same hello_world model.

```python

hello_ensemble.py

from smartsim import Experiment

exp = Experiment("helloworldbatch", launcher="auto")

define resources for all ensemble members

batch = exp.createbatchsettings(nodes=4, time="00:10:00", account="12345-Cray") batch.set_queue("premium")

define how each member should run

run = exp.createrunsettings(exe="echo", exeargs="Hello World!") run.settasks(60) run.settasksper_node(20)

ensemble = exp.createensemble("helloworld", batchsettings=batch, runsettings=run, replicas=4) exp.start(ensemble, block=True, summary=True)

print(exp.get_status(ensemble)) ```

bash python hello_ensemble.py

Similar to the interactive example, this same script will run on a SLURM, PBS, or SGE system as the launcher is set to auto in the Experiment initialization. Local launching does not support batch workloads.

Infrastructure Library Applications

• Orchestrator - In-memory data store and Machine Learning Inference (Redis + RedisAI)

Redis + RedisAI

The Orchestrator is an in-memory database that utilizes Redis and RedisAI to provide a distributed database and access to ML runtimes from Fortran, C, C++ and Python.

SmartSim provides classes that make it simple to launch the database in many configurations and optionally form a distributed database cluster. The examples below will show how to launch the database. Later in this document we will show how to use the database to perform ML inference and processing.

Local Launch

The following script launches a single database using the local launcher.

Experiment.create_database will initialize an Orchestrator instance corresponding to the specified launcher.

```python

rundblocal.py

from smartsim import Experiment

exp = Experiment("local-db", launcher="local") db = exp.create_database(port=6780, # database port interface="lo") # network interface to use

by default, SmartSim never blocks execution after the database is launched.

exp.start(db)

launch models, analysis, training, inference sessions, etc

that communicate with the database using the SmartRedis clients

stop the database

exp.stop(db) ```

Interactive Launch

The Orchestrator, like Ensemble instances, can be launched locally, in interactive allocations, or in a batch.

The following example launches a distributed (3 node) database cluster in an interactive allocation.

```bash

get interactive allocation (Slurm)

salloc -N 3 --ntasks-per-node=1 --exclusive -t 00:10:00

get interactive allocation (PBS)

qsub -l select=3:ncpus=1 -l walltime=00:10:00 -l place=scatter -I -q queue

```

```python

run_db.py

from smartsim import Experiment

auto specified to work across launcher types

exp = Experiment("db-on-slurm", launcher="auto") dbcluster = exp.createdatabase(dbnodes=3, dbport=6780, batch=False, interface="ipogif0") exp.start(db_cluster)

print(f"Orchestrator launched on nodes: {db_cluster.hosts}")

launch models, analysis, training, inference sessions, etc

that communicate with the database using the SmartRedis clients

exp.stop(db_cluster) bash

in interactive terminal

python run_db.py ```

Batch Launch

The Orchestrator can also be launched in a batch without the need for an interactive allocation. SmartSim will create the batch file, submit it to the batch system, and then wait for the database to be launched. Users can hit CTRL-C to cancel the launch if needed.

```Python

rundbbatch.py

from smartsim import Experiment

exp = Experiment("batch-db-on-pbs", launcher="auto") dbcluster = exp.createdatabase(dbnodes=3, dbport=6780, batch=True, time="00:10:00", interface="ib0", account="12345-Cray", queue="cl40")

exp.start(db_cluster)

print(f"Orchestrator launched on nodes: {db_cluster.hosts}")

launch models, analysis, training, inference sessions, etc

that communicate with the database using the SmartRedis clients

exp.stop(db_cluster) ```

bash python run_db_batch.py

SmartRedis

The SmartSim IL Clients (SmartRedis) are implementations of Redis clients that implement the RedisAI API with additions specific to scientific workflows.

SmartRedis clients are available in Fortran, C, C++, and Python. Users can seamlessly pull and push data from the Orchestrator from different languages.

Tensors

Tensors are the fundamental data structure for the SmartRedis clients. The Clients use the native array format of the language. For example, in Python, a tensor is a NumPy array while the C/C++ clients accept nested and contiguous arrays.

When stored in the database, all tensors are stored in the same format. Hence, any language can receive a tensor from the database no matter what supported language the array was sent from. This enables applications in different languages to communicate numerical data with each other at runtime.

For more information on the tensor data structure, see the documentation

Datasets

Datasets are collections of Tensors and associated metadata. The Dataset class is a user space object that can be created, added to, sent to, and retrieved from the Orchestrator.

For an example of how to use the Dataset class, see the Online Analysis example

For more information on the API, see the API documentation

SmartSim + SmartRedis Tutorials

SmartSim and SmartRedis were designed to work together. When launched through SmartSim, applications using the SmartRedis clients are directly connected to any Orchestrator launched in the same Experiment.

In this way, a SmartSim Experiment becomes a driver for coupled ML and Simulation workflows. The following are simple examples of how to use SmartSim and SmartRedis together.

Run the Tutorials

Each tutorial is a Jupyter notebook that can be run through the SmartSim Tutorials docker image which will run a jupyter lab with the tutorials, SmartSim, and SmartRedis installed.

bash docker pull ghcr.io/craylabs/smartsim-tutorials:latest docker run -p 8888:8888 ghcr.io/craylabs/smartsim-tutorials:latest Each of the following examples can be found in the SmartSim documentation.

Online Analysis

Using SmartSim, HPC applications can be monitored in real time by streaming data from the application to the database. SmartRedis clients can retrieve the data, process, analyze it, and finally store any updated data back to the database for use by other clients.

The following is an example of how a user could monitor and analyze a simulation. The example here uses the Python client; however, SmartRedis clients are also available for C, C++, and Fortran. All SmartRedis clients implement the same API.

The example will produce this visualization while the simulation is running.

Lattice Boltzmann Simulation

Using a Lattice Boltzmann Simulation, this example demonstrates how to use the SmartRedis Dataset API to stream data over the Orchestrator deployed by SmartSim.

The simulation will be composed of two parts: fv_sim.py which will generate data from the Simulation and store it in the Orchestrator, and driver.py which will launch the Orchestrator, start fv_sim.py and check for data posted to the Orchestrator to plot updates in real-time.

The following code highlights the sections of fv_sim.py that are responsible for transmitting the data needed to plot timesteps of the simulation to the Orchestrator.

```Python

fv_sim.py

from smartredis import Client import numpy as np

initialization code omitted

save cylinder location to database

cylinder = (X - xres/4)**2 + (Y - yres/2)2 < (y_res/4)2 # bool array client.put_tensor("cylinder", cylinder.astype(np.int8))

for time_step in range(steps): # simulation loop for i, cx, cy in zip(idxs, cxs, cys): F[:,:,i] = np.roll(F[:,:,i], cx, axis=1) F[:,:,i] = np.roll(F[:,:,i], cy, axis=0)

bndryF = F[cylinder,:]
bndryF = bndryF[:,[0,5,6,7,8,1,2,3,4]]

rho = np.sum(F, 2)
ux  = np.sum(F * cxs, 2) / rho
uy  = np.sum(F * cys, 2) / rho

Feq = np.zeros(F.shape)
for i, cx, cy, w in zip(idxs, cxs, cys, weights):
    Feq[:,:,i] = rho * w * ( 1 + 3*(cx*ux+cy*uy)  + 9*(cx*ux+cy*uy)**2/2 - 3*(ux**2+uy**2)/2 )
F += -(1.0/tau) * (F - Feq)
F[cylinder,:] = bndryF

# Create a SmartRedis dataset with vorticity data
dataset = Dataset(f"data_{str(time_step)}")
dataset.add_tensor("ux", ux)
dataset.add_tensor("uy", uy)

# Put Dataset in db at key "data_{time_step}"
client.put_dataset(dataset)

```

The driver script, driver.py, launches the Orchestrator database and runs the simulation in a non-blocking fashion. The driver script then uses the SmartRedis client to pull the DataSet and plot the vorticity while the simulation is running.

```Python

driver.py

time_steps, seed = 3000, 42

exp = Experiment("finitevolumesimulation", launcher="local")

db = exp.create_database(port=6780, # database port interface="lo") # network interface db should listen on

create the lb simulation Model reference

settings = exp.createrunsettings("python", exeargs=["fvsim.py", f"--seed={seed}", f"--steps={timesteps}"]) model = exp.createmodel("fvsimulation", settings) model.attachgeneratorfiles(tocopy="fv_sim.py") exp.generate(db, model, overwrite=True)

exp.start(db) client = Client(address=db.get_address()[0], cluster=False)

start simulation (non-blocking)

exp.start(model, block=False, summary=True)

poll until simulation starts and then retrieve data

client.pollkey("cylinder", 200, 100) cylinder = client.gettensor("cylinder").astype(bool)

for i in range(0, timesteps): client.pollkey(f"data{str(i)}", 10, 1000) dataset = client.getdataset(f"data{str(i)}") ux, uy = dataset.gettensor("ux"), dataset.get_tensor("uy")

# analysis/plotting code omitted

exp.stop(db) ```

For more examples of how to use SmartSim and SmartRedis together to perform online analysis, please see the online analsysis tutorial section of the SmartSim documentation.

Online Processing

Each of the SmartRedis clients can be used to remotely execute TorchScript code on data stored within the database. The scripts/functions are executed in the Torch runtime linked into the database.

Any of the functions available in the TorchScript builtins can be saved as "script" or "functions" in the database and used directly by any of the SmartRedis Clients.

Singular Value Decomposition

For example, the following code sends the built-in Singular Value Decomposition to the database and execute it on a dummy tensor.

```python import numpy as np from smartredis import Client

don't even need to import torch

def calcsvd(inputtensor): return input_tensor.svd()

connect a client to the database

client = Client(cluster=False)

get dummy data

tensor = np.random.randint(0, 100, size=(5, 3, 2)).astype(np.float32)

client.puttensor("input", tensor) client.setfunction("svd", calc_svd)

client.runscript("svd", "calcsvd", "input", ["U", "S", "V"])

results are not retrieved immediately in case they need

to be fed to another function/model

U = client.gettensor("U") S = client.gettensor("S") V = client.get_tensor("V") print(f"U: {U}, S: {S}, V: {V}") ```

The processing capabilities make it simple to form computational pipelines of functions, scripts, and models.

See the full TorchScript Language Reference documentation for more information on available methods, functions, and how to create your own.

Online Inference

Where available, SmartSim supports the following frameworks for querying Machine Learning models from C, C++, Fortran and Python with the SmartRedis Clients:

A number of other libraries are supported through ONNX, like SciKit-Learn and XGBoost.

A more detailed breakdown of supported toolkit versions for different platforms is available in the installation section of the documentation.

Note: It's important to remember that SmartSim utilizes a client-server model. To run experiments that utilize the above frameworks, you must first start the Orchestrator database with SmartSim.

PyTorch CNN Example

The example below shows how to spin up a database with SmartSim and invoke a PyTorch CNN model using the SmartRedis clients.

```python

simpletorchinference.py

import io import torch import torch.nn as nn from smartredis import Client from smartsim import Experiment

exp = Experiment("simple-online-inference", launcher="local") db = exp.create_database(port=6780, interface="lo")

class Net(nn.Module): def init(self): super().init() self.conv = nn.Conv2d(1, 1, 3)

def forward(self, x):
    return self.conv(x)

torchmodel = Net() exampleforwardinput = torch.rand(1, 1, 3, 3) module = torch.jit.trace(torchmodel, exampleforwardinput) modelbuffer = io.BytesIO() torch.jit.save(module, modelbuffer)

exp.start(db, summary=True)

address = db.get_address()[0] client = Client(address=address, cluster=False)

client.puttensor("input", exampleforwardinput.numpy()) client.setmodel("cnn", modelbuffer.getvalue(), "TORCH", device="CPU") client.runmodel("cnn", inputs=["input"], outputs=["output"]) output = client.get_tensor("output") print(f"Prediction: {output}")

exp.stop(db) ```

The above python code can be run like any normal python script: bash python simple_torch_inference.py

For more examples of how to use SmartSim and SmartRedis together to perform online inference, please see the online inference tutorials section of the SmartSim documentation.

Publications

The following are public presentations or publications using SmartSim

• Collaboration with NCAR - CGD Seminar
• SmartSim: Using Machine Learning in HPC Simulations
• SmartSim: Online Analytics and Machine Learning for HPC Simulations
• PyTorch Ecosystem Day Poster

Cite

Please use the following citation when referencing SmartSim, SmartRedis, or any SmartSim related work:

Partee et al., “Using Machine Learning at Scale in HPC Simulations with SmartSim: An Application to Ocean Climate Modeling”, Journal of Computational Science, Volume 62, 2022, 101707, ISSN 1877-7503

Available: https://doi.org/10.1016/j.jocs.2022.101707.

bibtex

latex @article{PARTEE2022101707, title = {Using Machine Learning at scale in numerical simulations with SmartSim: An application to ocean climate modeling}, journal = {Journal of Computational Science}, volume = {62}, pages = {101707}, year = {2022}, issn = {1877-7503}, doi = {https://doi.org/10.1016/j.jocs.2022.101707}, url = {https://www.sciencedirect.com/science/article/pii/S1877750322001065}, author = {Sam Partee and Matthew Ellis and Alessandro Rigazzi and Andrew E. Shao and Scott Bachman and Gustavo Marques and Benjamin Robbins}, keywords = {Deep learning, Numerical simulation, Climate modeling, High performance computing, SmartSim}, }